{"schema_version":"1.0","name":"Reific","website_operator":"Reific, Inc.","website_data_controller":"Reific, Inc.","corporate_entities":{"parent":{"name":"Reific, Inc.","legalForm":"Delaware corporation","register":"Delaware Division of Corporations","registrationNumber":"10740924","registeredOffice":"251 Little Falls Drive, Wilmington, Delaware 19808, United States","registeredAgent":"Corporation Service Company"},"subsidiary":{"name":"Reific, UAB","legalForm":"Lithuanian private limited liability company (uždaroji akcinė bendrovė)","register":"Register of Legal Entities of the Republic of Lithuania","registrationNumber":"308114457","registeredOffice":"A. Jakšto g. 6A-22, LT-01105 Vilnius, Lithuania","electronicDelivery":"308114457"},"details":"/company"},"description":"Turn operational time series into focused, source-linked evidence for AI agents. Find the relevant windows, preserve context, and inspect the cost of every answer.","url":"https://web-23297-afd22a2d-8mlq88ht.onporter.run","language":"en","environment":"unindexed_preview","updated":"2026-09-17","workflow":["An agent asks a bounded question about operational history.","Relevant time windows are selected with surrounding context and source references.","The agent receives a compact evidence packet and can inspect the underlying evidence.","Answers are evaluated for correctness, appropriate abstention, latency, and total cost."],"evidence_contract":{"contents":["question","time_window","selected_signals","source_references","limitations"],"claim_boundary":"The interactive visual illustrates the intended workflow. It is not a live customer trace or a measured performance result.","model_boundary":"Reific's owned learned temporal model is research in progress. This website does not establish its production readiness or general accuracy.","evaluation_boundary":"Public-data evaluations and illustrative calculations must be labeled as such. They are not anonymized customer results."},"measurement":{"token_reduction_formula":"1 - evidence_arm_total_tokens / baseline_arm_total_tokens","token_scope":"Count input and output tokens over every model call, including retries and tool-result context. Name the tokenizer and model.","cost_scope":"Include evidence preparation and retrieval, model input and output, retries, and amortized recurring processing. Token reduction alone is not total cost reduction.","resolution_rate_formula":"correctly_resolved_tasks / eligible_tasks","comparison_controls":"Use the same tasks, agent model, tools, answer rubric, and task budget. Include full context and a competent analysis-tool baseline."},"founder_reported_benchmarks":{"status":"founder_reported_supporting_records_pending","reported_at":"2026-09-15","source":"Founder confirmation during the website preview review.","independently_verified_for_this_preview":false,"reported_resolution_improvement_percent":50,"reported_token_burn_reduction_percent":40,"disclosure":"The founder reports 50% better resolution and 40% less token burn from separate benchmarks. Supporting run records and methodology are not attached to this preview.","benchmark_tasks":null,"agent_model":null,"methodology":null,"run_records":null,"baseline_resolution_rate":null,"reific_resolution_rate":null,"resolution_improvement_basis":null,"baseline_total_tokens":null,"reific_total_tokens":null,"token_accounting_scope":null,"unknowns":"Task set, model, scoring method, absolute resolution and token counts, token-accounting scope, and the relative-versus-percentage-point basis of the resolution claim have not been supplied.","scope_boundary":"These reported benchmarks are separate from the deterministic public-data replays below. The replays measure input-payload sizes and make zero model calls; they do not establish agent resolution or total token burn.","arithmetic_example":{"status":"hypothetical_arithmetic_not_benchmark_counts","resolution_interpretation":"relative increase for this worked example","baseline_resolution_rate":0.4,"resulting_resolution_rate":0.6,"relative_resolution_improvement_percent":50,"resolution_increase_percentage_points":20,"baseline_tokens":100000,"resulting_tokens":60000,"token_reduction_percent":40}},"measured_evidence":{"status":"reference_replay_payload_only","scope":"Exact serialized data payload tokens only. No language model, paid API, task success, customer outcome, wall-clock advantage, or end-to-end cost was measured. Projection is a stronger simple baseline; the deterministic computations themselves can also run in SQL or Python.","tokenizer":"tiktoken==0.11.0","encoding":"o200k_base","study_count":2,"projected_input_tokens":150801,"evidence_input_tokens":1444,"payload_reduction_percent":99.0424,"agent_calls":0,"agent_resolution_rate":null,"total_agent_token_burn":null,"replay":"/evidence/replay.json"},"articles":[{"slug":"what-agents-need-from-time-series","title":"What an agent needs to answer a time-series question","url":"/research/what-agents-need-from-time-series","kind":"field-note","date":"2026-09-17","updatedAt":"2026-09-17","author":"Reific","related":["sql-rag-and-time-series-models","building-demand","what-counts-as-a-correct-answer"],"publications":[],"markdown":"/research/what-agents-need-from-time-series/markdown","summary":"A useful answer connects a precise question to the right readings, their meaning, and the sources that let someone check the conclusion. A public building-energy example makes that contract concrete.","question":"What should we give an AI agent so it can answer questions about operational time-series data?","sections":[{"title":"The question decides which readings matter","paragraphs":["An agent answering a question about a time series needs a way to find relevant measurements, understand what they represent, and connect them to a conclusion someone can inspect. Giving it access to a database starts that process. The harder design work is deciding what information an answer must preserve.","Consider a request to find the biggest energy spike in a building. That could mean the highest individual reading, the fastest increase, or the greatest consumption over a sustained period. Each interpretation can select a different event. Before discussing models, we need to turn that request into a calculation with a clear unit and time window.","For our public example, we use the UCI Appliances Energy Prediction dataset. It records appliance energy in watt-hours at ten-minute intervals, alongside other building and weather measurements. We ask a specific question: which three consecutive readings have the highest combined appliance energy in the first seven complete days? This fixes what we will compare and what would count as a correct numerical answer. [1]"]},{"title":"One week becomes an inspectable event","paragraphs":["The selected week contains 1,008 readings, from 12 through 18 January 2016. Our reference script checks all 1,006 groups of three consecutive readings. The largest sum is 560 + 1,080 + 750 = 2,390 Wh, recorded at 18:40, 18:50, and 19:00 on 16 January. The evidence includes those values and their locations in the original file: CSV lines 732, 733, and 734, counting the header as line one. [2]","There is a small but consequential detail in that answer. The timestamp labels span twenty minutes, while three ten-minute energy readings represent thirty minutes of measured consumption. The source does not specify whether each timestamp marks an interval's start or end, or provide a timezone. We retain its labels and state the interval convention as unresolved. [2]","The three readings let a reader check the addition. The claim that this is the week's maximum also depends on the full scan. An inspectable answer therefore needs both the selected evidence and a record of the comparison that selected it. A cropped chart alone cannot establish that no larger window was excluded."],"figure":"evidence-window","figureDescription":"Building-energy replay: from 1,008 ten-minute readings, an exhaustive scan selects 560, 1,080 and 750 Wh at source timestamps 18:40, 18:50 and 19:00 on 16 January 2016. Sum: 2,390 Wh. CSV lines 732–734. These values answer which three intervals have the greatest sum; aggregate readings alone cannot identify the appliance or cause. The diagram demonstrates the evidence interface, not a live agent run."},{"title":"Preserve the meaning of the measurement","paragraphs":["A value of 1,080 means little without its unit. Watt-hours measure energy; watts measure power. A sensor's instantaneous output, an interval total, and a cumulative meter reading require different calculations. Summing three values is appropriate for the energy intervals in this example. Applying the same operation to three cumulative meter readings would answer a different, usually unhelpful question.","The entity matters just as much. These are aggregate appliance-energy measurements. They do not identify which appliance was running. A useful answer keeps that distinction visible even if the reader asks about a particular device. Otherwise a correct calculation can become the foundation for an unsupported diagnosis.","Coverage deserves similar care. Our selected week has a consistent ten-minute cadence and no missing appliance values. In another record, three neighboring rows could straddle a gap. An agent needs that gap surfaced before treating the rows as a continuous period. Data quality is part of the answer because it determines which comparisons are valid. [2]"]},{"title":"Make the output smaller without weakening the evidence","paragraphs":["Sending every reading to a language model is one possible interface. A more focused interface can perform the numerical work first and return the result with enough context to examine it. For this question, the useful output contains the selected week, the comparison performed, the three readings, their source locations, and the limits of the conclusion.","Our two-column timestamp-and-energy CSV contains 16,838 tokens. The complete evidence JSON contains 529, measured with tiktoken 0.11.0 and the o200k_base encoding. These are counts of the serialized files. The replay makes zero language-model calls, so it measures neither answer quality nor total agent spending. A SQL query or Python script can compute the same maximum. [2]","The important design question is what survives the reduction. A sentence saying that energy use was unusually high would remove the calculation and introduce a judgment we have not established. The compact evidence retains a narrower, checkable claim. An agent can inspect the full source when the next question needs more context."]},{"title":"Keep the decision time separate from the event time","paragraphs":["This example scans a completed week. A live decision introduces another boundary: what information had arrived when the decision was made? Suppose a maintenance record describes work on Tuesday but is entered on Thursday. It may help explain Tuesday's measurements in a later investigation. It could not have informed an agent answering on Wednesday unless the information was available elsewhere.","This is a practical data issue. Feast's documentation distinguishes a feature's event timestamp from the creation timestamp used to constrain when the value was available. Filtering only by event time can admit later corrections into an earlier retrieval. The same distinction matters when constructing evidence for an operational agent. [4]","A retrospective explanation and an answer produced during an event can both be useful. Their evidence needs different time boundaries. An evaluation should preserve the boundary relevant to the actual job rather than quietly supplying later knowledge."]},{"title":"Give the agent a useful next question","paragraphs":["The energy event is established. Its cause is open. The selected maximum does not show that the building was wasteful or faulty. Device-level measurements, operating schedules, or intervention records could support a more specific investigation. Naming the missing evidence gives the agent somewhere productive to go.","This is the interface we are working toward at Reific: a precise question produces relevant readings with enough context and source access for an agent to form a bounded answer. Anthropic's guidance on tool design makes a related point: tool responses should provide useful context, and changes to those tools should be tested on realistic tasks. [3]","Our public replay makes that interface tangible. Demonstrating a product advantage requires a further comparison against an agent with appropriate query tools, using the same questions and information. The result to look for is a better complete investigation: correct readings, supported conclusions, useful requests for missing evidence, and less work left for the person checking the answer."]}],"sources":[{"title":"Candanedo — Appliances Energy Prediction, UCI Machine Learning Repository","href":"https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction"},{"title":"Reific — Building-energy replay, calculation, source files, and measurement limits","href":"/research/building-demand"},{"title":"Anthropic — Writing effective tools for agents","href":"https://www.anthropic.com/engineering/writing-tools-for-agents"},{"title":"Feast — Point-in-time joins and data availability","href":"https://docs.feast.dev/getting-started/concepts/point-in-time-joins"}]},{"slug":"sql-rag-and-time-series-models","title":"SQL, retrieval, and forecasting answer different questions","url":"/research/sql-rag-and-time-series-models","kind":"field-note","date":"2026-09-17","updatedAt":"2026-09-17","author":"Reific","related":["what-agents-need-from-time-series","building-demand","what-counts-as-a-correct-answer"],"publications":[],"markdown":"/research/sql-rag-and-time-series-models/markdown","summary":"One operational dataset can support a lookup, a comparison, an investigation, or a forecast. The question determines the method—and the evidence needed to trust its result.","question":"When should an agent use SQL, retrieval, or a time-series model to answer a question?","sections":[{"title":"Begin with the question, then choose the operation","paragraphs":["A team connecting an agent to operational data will soon face a choice between SQL, retrieval, and time-series models. It helps to make that choice at the level of a question. The same measurements can support several kinds of work, and one investigation may require all three methods.","Take the public building-energy record used in our reference replay. It contains appliance energy at ten-minute intervals. We can ask what the reading was at a particular time, whether usage changed, what might explain a selected event, or how much energy the building might use tomorrow. Those requests share a dataset. They do not share a definition of a correct answer. [1]","Separating them early prevents a common design mistake: expecting whichever component is easiest to connect to perform every part of the analysis. A search result may supply relevant context while leaving the arithmetic unresolved. An accurate forecast may still leave the cause of a past event unknown."],"figure":"question-methods","figureDescription":"Four questions require different operations: a past total needs a filtered numerical aggregation; a change needs an explicit comparison; an explanation needs corroborating operational records; a future event needs an evaluated forecast. Retrieval can support each, but a source match alone does not validate arithmetic or causation. This is a conceptual guide, not a Reific architecture diagram."},{"title":"An exact question should produce an exact calculation","paragraphs":["For a request such as the appliance-energy reading at 18:50 on 16 January 2016, the core operation is a lookup. Select the right entity and timestamp, return the value and unit, and preserve a link to the source. An agent may translate the request into a query, but the database should supply the number.","Our replay asks a slightly richer numerical question: find the three consecutive readings with the largest sum in the selected week. A window function or a short script can enumerate the candidate groups. The recorded maximum is 2,390 Wh across readings of 560, 1,080, and 750 Wh. No language model is needed for that calculation. [2]","Query correctness still depends on the data contract. A three-row window means three intervals only after checking ordering, duplicates, and sampling gaps. Energy and power need different aggregations. The selected period must be explicit. These details belong in the tool's validation and returned evidence, so the agent does not have to guess them from a column name."]},{"title":"A change needs a reference","paragraphs":["Now ask whether energy usage increased. We need a comparison: the previous reading, the previous hour, the same time yesterday, or a baseline adjusted for operating conditions. Choosing among them is part of defining the question. It should be visible in the answer.","A largest-value query and a largest-change query can select different periods. The highest energy total may be sustained, expected operation. A sharp rise from a low starting point may remain small in absolute terms. Calling either event abnormal introduces an additional standard: what range of behavior should be expected under comparable conditions?","Simple differences, rolling statistics, and matched historical comparisons are useful starting points. More elaborate detectors need an evaluation showing that their extra complexity helps with the intended task. Preserve the readings around the event and the baseline used. A conclusion about a change is much easier to review when the reader can see both sides of the comparison."]},{"title":"An explanation needs context the meter may not contain","paragraphs":["The next question is often why the selected event occurred. An aggregate energy meter cannot tell us which appliance ran or why someone switched it on. More analysis of the same three values will not create a missing operating record. At this point, the investigation needs additional sources.","Retrieval-augmented generation (RAG) supplies retrieved material to a language model as context for its answer. Retrieval can locate schedules, equipment descriptions, maintenance notes, or earlier incidents. Structured queries can connect those records to the right building and period. Similar wording alone is weak evidence: a note about another building, or about work completed after the event, may be highly relevant to the search phrase and irrelevant to this explanation.","OpenAI's account of its internal data agent offers a concrete example of combining these operations. The system uses table metadata, expert descriptions, organizational knowledge, and live queries. It evaluates generated SQL against expected results and lets users inspect the underlying data. That combination addresses several distinct sources of error. [3]","For the building replay, a schedule would be a candidate source to investigate; we have not supplied one or established a device-level cause. If an explanation relies on an assumed schedule, it should state the assumption and request confirmation. A retrieved document becomes evidence only to the extent that it supports the particular claim being made."]},{"title":"A forecast answers a question about the future","paragraphs":["Asking about tomorrow's energy use changes the task again. We need an issuance time, a forecast horizon, a target, and a definition of acceptable error. A forecast may use the building's history and information already available about tomorrow, such as an issued schedule. An evaluation must keep later observations outside that input.","A useful starting comparison might repeat a recent value or a comparable historical period. A time-series model earns its place by improving the relevant forecast against an appropriate baseline. Toto 2.0 is one public example of a model evaluated for forecasting across several benchmarks; those results concern predicted values under specified tests. They do not establish the cause of a particular building event. [4]","The methods can also combine. Research on retrieval-augmented time-series forecasting investigates using related historical sequences to help a forecasting model. Retrieval is an operation that can support prediction as well as explanation. Choosing a forecasting model does not require excluding retrieval from the system. [5]","Even when a model identifies a variable as useful for prediction, that does not by itself show what would happen if someone changed it. Physical cause and the effect of an intervention require further evidence. Keep that boundary visible when moving from a forecast to a recommendation."]},{"title":"Connect the methods through evidence someone can inspect","paragraphs":["A practical investigation might begin with a query, use a comparison to locate a change, retrieve a relevant operating record, and ask a forecasting model about the next period. Each step should return an identifiable result, its scope, and the source or method supporting it. The final answer can then explain how those pieces fit together without pretending that they establish more than they do.","At Reific, we are working on making operational history usable in that process. The useful interface is a bounded question followed by relevant readings, context, and sources that an agent can inspect. It should also make missing evidence apparent, so the next step follows from the unresolved question.","Start with the simplest complete workflow for the actual job. Check numerical queries against known results, compare changes under an explicit baseline, review explanations for support, and evaluate forecasts on unseen periods. Add a component when it improves that workflow. A sophisticated diagram cannot substitute for an answer whose values and conclusions survive inspection."]}],"sources":[{"title":"Candanedo — Appliances Energy Prediction, UCI Machine Learning Repository","href":"https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction"},{"title":"Reific — Building-energy replay and reproducible calculation","href":"/research/building-demand"},{"title":"OpenAI — Inside our in-house data agent","href":"https://openai.com/index/inside-our-in-house-data-agent/"},{"title":"Datadog — Toto 2.0: Time series forecasting enters the scaling era","href":"https://www.datadoghq.com/blog/ai/toto-2/"},{"title":"Retrieval Augmented Time Series Forecasting","href":"https://arxiv.org/abs/2411.08249"}]},{"slug":"what-counts-as-a-correct-answer","title":"What counts as a correct answer?","url":"/research/what-counts-as-a-correct-answer","kind":"field-note","date":"2026-09-15","updatedAt":"2026-09-17","author":"Reific","related":["evaluating-an-evidence-layer","measuring-agent-token-cost","room-occupancy"],"publications":[],"markdown":"/research/what-counts-as-a-correct-answer/markdown","summary":"An operational explanation must survive checks on the readings, the conclusion and the proposed action. Here is how to evaluate those separately, including when an agent should say it does not know.","question":"How do you evaluate whether an agent’s interpretation of time-series data is correct and useful?","sections":[{"title":"Begin with a claim someone can check","paragraphs":["Consider an illustrative machine investigation. Vibration increases, then temperature rises. An agent describes the sequence accurately and concludes that a bearing has failed. It recommends replacing the bearing. The paragraph is clear, uses the right engineering vocabulary and cites a chart. How much of it should pass an evaluation?","The measurements may be correct while the diagnosis remains unconfirmed. The proposed repair requires another judgment. A single score for whether the paragraph sounds like an engineer’s answer conceals these differences. It also makes improvement difficult: the team cannot tell whether to fix the data selection, the interpretation or the recommendation.","For an operational task, we would begin by agreeing what the answer must establish. Identify the machine, the decision and the information available to the person making it. Then evaluate each consequential claim against that contract. The example here illustrates this method; it is not a reported diagnosis or a Reific performance result."]},{"title":"Separate the parts of a useful answer","paragraphs":["We propose five checks: observation, interpretation, action, uncertainty and review effort. Observation covers the entity, interval, units, values and ordering. Interpretation asks whether the proposed condition is established, merely consistent with the readings or contradicted by another source. Action asks whether the next step follows from the evidence and the permitted task. Uncertainty records what remains unresolved. Review effort measures what a person must still verify or correct.","Each check needs its own evidence. A chart showing a vibration increase supports that observation. It does not independently verify a bearing defect. A reference explanation containing the same technical terms cannot settle the question either: two answers can be linguistically similar while naming different components or reversing the order of events.","Anthropic’s evaluation guidance distinguishes code, model and human graders. [1] Applied here, arithmetic and source references can often be checked programmatically; a domain expert should establish the interpretation and action criteria. An LLM judge can help apply a rubric, but agreement with that judge is not a substitute for checking what happened."],"figure":"answer-review","figureDescription":"Illustrative machine investigation: a vibration rise is an observation to verify against values and units; a proposed bearing fault needs independent corroboration; an inspection recommendation needs a proportionate action and accountable owner. An unresolved cause should stay unresolved. Score each claim separately; this is a proposed rubric, not measured performance."},{"title":"A recommendation needs its own evidence","paragraphs":["In the machine example, a reviewer might accept both signal changes and reject the bearing diagnosis. That need not make the answer useless. It could still justify a narrower next step: compare operating conditions, inspect relevant maintenance history or request the measurement that distinguishes competing explanations. The task owner should decide which of those steps is appropriate before the evaluation begins.","Grade recommendations at the level of authority the agent actually has. An assistant asked to prepare an investigation should not receive extra credit for recommending an intervention it cannot justify. Conversely, an evaluation should not require a definitive cause when the available evidence can only support referral to an engineer.","Keep consequential errors visible. Correct units and polished writing should not average away a recommendation aimed at the wrong asset. Define which mistakes invalidate the answer, then use the remaining checks to explain why it failed and what useful work it nevertheless completed."]},{"title":"Abstention has two failure modes","paragraphs":["An agent can fail by asserting too much. It can also fail by declining a question the supplied evidence answers. If the task asks which signal changed first, refusing because the root cause is unknown is an unnecessary abstention. If it asks which component must be replaced, the same missing cause may be decisive.","This is related to the risk–coverage tradeoff studied in selective classification: rejecting some cases changes both the error rate on answered cases and the fraction of cases answered. [2] An operational evaluation should therefore report accepted answers, unsupported conclusions, justified abstentions and avoidable abstentions separately. A lower error rate achieved by answering almost nothing may leave the whole workload with the engineer.","Make an abstention useful enough to review. It should identify the unresolved claim, the missing evidence and, where possible, a specific next check. Include cases where the record is sufficient as well as cases with missing, conflicting or irrelevant context. Otherwise the evaluation rewards one habitual response."]},{"title":"Preserve what was knowable at the time","paragraphs":["Historical cases are convenient because a later outcome may already be known. They also create an easy route for leakage. In a second illustrative example, a fault event occurs at 09:12, but its record reaches the investigation system at 09:24. An evaluation of a 09:20 decision must exclude that record. Filtering only on the event timestamp would incorrectly admit it.","Forecast evaluation has an established temporal discipline: the forecast must be constructed without future observations. [3] For an operational investigation, we extend that discipline to availability time, document revisions and later corrections. A repair finding recorded the next day can help grade the diagnosis; it cannot be supplied as evidence the agent supposedly had the previous morning.","Retain the input snapshot, the decision cutoff and the basis for every outcome label. Where arrival times or historical document versions are unavailable, state that the case cannot establish performance under the original information constraints. A retrospective explanation can still be useful, but it answers a different question."],"figure":"decision-time","figureDescription":"Illustrative point-in-time evaluation: a fault occurs at 09:12, a decision is made at 09:20, a delayed record arrives at 09:24, and a repair finding is recorded the next day. The late record and repair finding are excluded from the 09:20 input; later evidence can support retrospective scoring. Event time alone does not establish availability."},{"title":"Compare against a capable baseline","paragraphs":["The comparison should give an ordinary agent appropriate query and code tools. For a question about a maximum, a join or a before-and-after interval, those tools may solve the task directly. Comparing an evidence layer only against an agent forced to read a large raw table would leave its practical contribution unresolved.","Hold the questions, eligible information, tool permissions and grading criteria fixed. Record the model and configuration used by each approach. Separate cases used to refine the system from the cases used to report its performance, and keep related windows from the same incident together when assigning those groups.","Research benchmarks can help isolate specific capabilities. ARFBench, for example, evaluates questions about time-series anomalies in software incident data. [4] Our application of that distinction is straightforward: a component score should be described at the task it measures. It does not, by itself, establish that an agent can choose an appropriate intervention or reduce an engineer’s workload."]},{"title":"Report the work left to the reviewer","paragraphs":["An evaluation report should let someone reconstruct a result: attempted cases, accepted cases, failure types, abstentions, source checks and expert corrections. Show the difficult examples alongside the successful ones. If reviewers disagree, retain that disagreement and resolve the rubric before interpreting small score differences.","This is the standard we want an evaluation of Reific to meet. The output should help a person reach a justified conclusion with less repeated investigation. Establishing that requires both a correct answer and a record of the work still needed to trust it."]}],"sources":[{"title":"Anthropic — Demystifying evals for AI agents","href":"https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents"},{"title":"Geifman and El-Yaniv — Selective Classification for Deep Neural Networks","href":"https://arxiv.org/abs/1705.08500"},{"title":"Hyndman and Athanasopoulos — Time series cross-validation","href":"https://otexts.com/fpp3/tscv.html"},{"title":"Xie et al. — ARFBench: Time Series Question Answering for Software Incident Response","href":"https://arxiv.org/abs/2604.21199"}]},{"slug":"measuring-agent-token-cost","title":"Less context. What did we actually save?","url":"/research/measuring-agent-token-cost","kind":"field-note","date":"2026-09-15","updatedAt":"2026-09-17","author":"Reific","related":["building-demand","evaluating-an-evidence-layer","what-counts-as-a-correct-answer"],"publications":[],"markdown":"/research/measuring-agent-token-cost/markdown","summary":"A smaller tool response is easy to measure. A cheaper, equally useful investigation takes a more careful comparison. Here is the calculation, from a saved file to an annual estimate.","question":"How do you measure AI agent token savings, data volume, and annual cost?","sections":[{"title":"Start with something another person can count","paragraphs":["An agent investigating a week of building measurements could receive every row, two relevant columns, or the readings around a particular event. Those inputs have different sizes. That is a useful observation, but it leaves the harder question open: how much work did the entire investigation require?","Our building-energy replay gives the comparison a concrete starting point. It finds the three consecutive ten-minute energy readings with the largest sum. The saved two-column CSV contains 16,838 tokens. The complete evidence packet contains 529, including its question, source references, selected readings, and limits. Both files are available to inspect and reproduce.[1]","The count uses tiktoken 0.11.0 with the o200k_base encoding on the exact UTF-8 text. Headers, delimiters, whitespace, and the trailing newline are included. A tokenizer describes a particular representation; it does not turn a CSV measurement into a provider invoice.[2]"]},{"title":"Choose the baseline before calculating the percentage","paragraphs":["The arithmetic is (16,838 − 529) ÷ 16,838 × 100 = 96.8583% less input payload. The unprojected, 29-field CSV is 250,068 tokens. Comparing against that file would produce a larger percentage, but much of the gain would come from removing columns an ordinary query could already exclude.","Even the two-column projection is not the strongest baseline for this question. A short SQL query or Python calculation can return the same maximum without sending the whole column to a language model. The replay therefore demonstrates a compact, inspectable result. It makes zero model calls and does not measure an agent's resolution rate or its total bill.","This is a practical design constraint for Reific: an evidence layer should be compared with an agent that can use appropriate tools. A large raw input is a useful stress test, but it should not be the only alternative in a savings claim. Anthropic's tool-design guidance similarly treats useful response content, token consumption, and task outcomes as things to evaluate together.[3]"],"figure":"token-ledger","figureDescription":"Exact building-replay text counts: full 29-field CSV 250,068 tokens; timestamp and energy projection 16,838; complete evidence JSON 529. The latter saves 16,309 tokens, or 96.8583% of the projected payload, using tiktoken 0.11.0 and o200k_base. No model calls. A complete run must separately count preparation, all model calls, source queries, retries, and verification; payload size is not billed savings."},{"title":"Count the run, including the work that moved elsewhere","paragraphs":["A compact first response can lead to a long investigation. The agent may reopen the source, retrieve another interval, retry a failed query, or ask a second model to check the answer. Conversely, a larger response may let it finish in one pass. Save each request's usage record and associate every call with the task that caused it.","Count input and output across all calls, including evidence preparation when it uses a model. Keep cached and uncached input separate when their prices differ. Use the provider's reported accounting for reasoning and other token categories without adding a subcategory twice. Failed and timed-out tasks remain in the ledger; otherwise a system can look cheaper by failing early.","Then account for costs outside the model: source queries, deterministic processing, storage, network transfer, and any recurring preprocessing needed to keep the evidence current. Shared work needs an explicit allocation rule. If an index serves several workflows, charging its whole cost to one task is as misleading as charging none of it.","Quality belongs next to this ledger. Under a fixed rubric, record accepted answers, unsupported answers, justified abstentions, and human corrections. Cost per accepted task can be useful, but show total cost and the number of attempted tasks too. A low price achieved by avoiding difficult questions is visible only when the denominator survives."]},{"title":"Data volume describes the workload, not the token count","paragraphs":["Gigabytes matter because they describe what the system must process. They do not determine how many tokens an agent consumes. A compressed sensor file, a verbose JSON export, and a binary array can represent the same measurements with different byte sizes. Querying an aggregate can return a small result from a large store.","For a planning estimate, use data examined per day divided by average data examined per comparable task. Measure both volumes in the same representation and include repeat reads consistently. Ten gigabytes per day divided by 100 megabytes per task gives 100 estimated tasks. These are decimal units: 1 GB = 1,000 MB. Actual task counts are preferable whenever they are available.","Suppose those tasks currently consume 20,000 total tokens each. The estimate is 2,000,000 tokens per day. Applying a hypothetical 40% reduction gives 1,200,000, or 800,000 fewer. This assumes that task mix and tokens per task stay representative as volume changes; it does not establish throughput or a universal bytes-to-tokens conversion."]},{"title":"From tokens to an annual estimate","paragraphs":["Annual model-token savings equal saved tokens per day × active days per year × price per million tokens ÷ 1,000,000. With 800,000 saved tokens, 365 active days, and an illustrative blended price of 5 currency units per million, the estimate is 1,460 per year. Model-token spend falls from 3,650 to 2,190 under those assumptions.","A blended price should come from representative billed runs: divide model-token charges by billed tokens and multiply by one million. Using the same price before and after assumes the mixture of input, output, and cache usage remains stable. When it changes, price those categories separately. Vendor fees, infrastructure, taxes, and review effort belong in a net-cost comparison; the token estimate alone is not net ROI.","The landing-page calculator lets you change workload, active days, price, and currency. Enter a price in the currency you select: switching between USD and EUR does not apply an exchange rate. The starting price is illustrative, not a quoted provider rate."]},{"title":"Keep the reported result and the worked example distinct","paragraphs":["Across separate benchmarks, Reific's founder reports 50% better resolution and 40% less token burn. The calculations here explain the arithmetic; they do not reproduce those benchmark runs. The public-data replay and illustrative workload are separate. The replay's 96.8583% figure measures serialized input size only.","For the calculator, “50% better” is interpreted as a relative increase: a 40% baseline becomes 60%, a gain of 20 percentage points. The starting rates are illustrative. A measured comparison should publish accepted and attempted task counts, the comparison basis, model and tool versions, complete usage, and failure cases alongside its percentage.","To test the savings on your own workflow, begin with a small set of real questions and keep both arms' complete records. Inspect which steps disappeared, which moved into preprocessing, and which still require an expert. That is the evidence needed to turn a smaller payload into a useful operating decision."]}],"sources":[{"title":"Reific — building-energy replay, downloadable inputs and reproduction script","href":"/research/building-demand"},{"title":"OpenAI — tiktoken, explicit encodings and token counting","href":"https://github.com/openai/tiktoken"},{"title":"Anthropic — Writing effective tools for agents (2025)","href":"https://www.anthropic.com/engineering/writing-tools-for-agents"}]},{"slug":"evaluating-an-evidence-layer","title":"When does an evidence layer earn its place?","url":"/research/evaluating-an-evidence-layer","kind":"field-note","date":"2026-09-17","updatedAt":"2026-09-17","author":"Reific","related":["what-counts-as-a-correct-answer","measuring-agent-token-cost"],"publications":[],"markdown":"/research/evaluating-an-evidence-layer/markdown","summary":"A fair pilot compares direct tools, the platform you already use and an agent with Reific on the same work. Define the comparison before spending engineering time on integration.","question":"When should a team use direct tools, its existing platform or a separate evidence layer?","sections":[{"title":"Start with the investigation that keeps recurring","paragraphs":["A team evaluating an evidence layer is already paying for an investigation process. Someone chooses a time window, finds the relevant entities, queries their histories, compares events and checks the conclusion. Some of that work may be automated. The useful starting point is to identify which part still requires repeated engineering effort and whether it matters enough to change.","Take an illustrative task: at 09:20, which machines need an engineer’s review, and what observations support each referral? An answer needs a bounded set of candidates, source readings and a defensible reason for inclusion. The task is more specific than asking an agent to find insights. It also gives the evaluator a way to recognize a good answer from any implementation.","A pilot should test the proposed improvement to that job. It should establish whether the additional component changes answer quality, review effort or cost enough to justify its continuing maintenance. A successful demonstration alone does not answer that question."]},{"title":"Give each approach a fair chance","paragraphs":["Begin with direct tools. An agent can query a database, calculate changes in code and retrieve specific events. Prometheus, for example, exposes expression queries over a specified time range and resolution through its HTTP API. [1] If an adequately scoped query supplies everything needed, adding another service may bring little benefit. That is a useful result to discover early.","The second approach is the platform already responsible for the workflow, including its current investigation features and the team’s own extensions. Established platforms can be substantial alternatives. Datadog describes both an incident-investigation agent and infrastructure for replaying and evaluating its behavior. [2] Evaluate the actual configured system your engineers would use, with the access and context that make it effective.","The third approach is the same agent workflow with Reific supplying evidence. This is a comparison to run, not an assumed ranking. Its case for adoption is whatever additional value the evaluation establishes after preparation, retrieval, interpretation and review are all counted. Keep the direct-tool and platform approaches in the comparison even when they answer some cases easily; those cases are part of the workload too."],"figure":"pilot-comparison","figureDescription":"Proposed paired evaluation: direct query/code tools, an existing platform workflow, and an agent with Reific receive the same questions, source access and time cutoff. Compare accepted outcomes, unsupported answers, abstentions, human corrections, elapsed time, and complete cost. The diagram contains no measured winner or vendor capability score."},{"title":"Write the acceptance contract before integration","paragraphs":["The person responsible for accepting the result should define the question and rubric. For the machine-review example, that means agreeing which observations warrant referral, which sources support them, what counts as a missed case and when uncertainty is acceptable. If the historical record cannot support a reliable answer, mark that case accordingly. Do not quietly turn uncertain maintenance notes into confirmed labels.","Choose cases that represent the intended workload: routine questions, relevant changes, apparent changes explained by operating conditions and incomplete records. Preserve the information available at the original decision time. Keep a development set for fixing integration and a separate assessment set for the comparison. Group related cases so nearly identical windows from one incident do not appear on both sides.","Agree the meaningful improvement before opening assessment results. That might be fewer consequential errors without more missed cases, or less reviewer time at an unchanged acceptance standard. Define the allowed latency, spending and preparation effort too. The thresholds should come from the job’s economics and error costs, with an explicit owner who can decide what the result permits next."]},{"title":"Make the comparison paired and inspectable","paragraphs":["Run each approach on every eligible assessment case. Use the same question, decision cutoff, underlying information and outcome rubric. For the two agent configurations, hold the downstream model, prompt and tool permissions constant wherever possible. The changed input is the evidence treatment being evaluated. Let the direct-tool agent select and aggregate data rather than forcing it to consume a full export.","An existing platform may use models or internal context you cannot hold constant. Record that difference. Its result is a comparison of complete workflows; it cannot isolate the effect of one evidence component. Both comparisons are useful, provided the report says which question each answers.","Fix the number of attempts in advance and retain every attempt. Anthropic notes that agent behavior can vary across trials. [3] Report both the paired case results and variation between attempts, without selecting the best answer from one approach and the first from another. Treat repeated trials and related windows as related observations, not additional independent incidents. Have reviewers inspect outputs without implementation names where practical, and preserve the grading rationale."]},{"title":"Account for the work around the model","paragraphs":["The cost ledger begins before the first answer. Record the engineering time spent mapping entities, preparing data, configuring permissions and repairing integration problems. During each run, retain requests, tool results, model usage, elapsed time, failures and retries. Include evidence preparation and service charges alongside the agent’s own consumption.","Keep one-time setup separate from recurring effort. A reusable mapping may be a reasonable investment; a mapping rebuilt manually for every case is part of the continuing operating cost. Treat reviewer time the same way. Track how long it takes to verify sources, correct a conclusion and produce an acceptable final answer, rather than timing only the generated response.","Inspect the differences case by case. Fewer model tokens may coexist with slower retrieval. A more accurate first answer may still require more expert review. A cheap workflow may omit hard cases. Report these tradeoffs with the workload denominator intact, then apply the acceptance contract instead of choosing whichever metric makes a preferred system look strongest."]},{"title":"Require a result that can be used again","paragraphs":["Before expanding the integration, repeat the agreed procedure on another reserved batch representing the intended work. This checks whether the setup and review process can be reused, within the limits of the sample. It also exposes dependencies a carefully prepared demonstration can hide: who notices a missing source, who updates a changed field and who investigates a failed run.","The decision may be to keep the direct tools, use the existing platform more fully or continue with Reific on a narrower class of tasks. An inconclusive result should identify the missing evidence and whether obtaining it is worth the effort. A promising offline result can justify a bounded trial in the real workflow; it does not yet establish recurring operational value.","For Reific to earn its place, the comparison should show a useful difference that survives those conditions. The team should be able to inspect where the difference came from, understand the remaining failures and estimate the work required to keep it. That is a more durable basis for adoption than the breadth of a product description."]}],"sources":[{"title":"Prometheus — HTTP API and range queries","href":"https://github.com/prometheus/prometheus/blob/main/docs/querying/api.md#range-queries"},{"title":"Datadog — Building a real-world evaluation platform for autonomous SRE agents","href":"https://www.datadoghq.com/blog/engineering/bits-ai-eval-platform/"},{"title":"Anthropic — Demystifying evals for AI agents","href":"https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents"}]},{"slug":"everest-rare-event-forecasting","title":"EVEREST: when the rare event is the one that matters","url":"/research/everest-rare-event-forecasting","kind":"research","date":"2026-09-17","updatedAt":"2026-09-17","author":"Reific","related":["what-counts-as-a-correct-answer","what-agents-need-from-time-series"],"publications":[{"title":"EVEREST: A Transformer for Probabilistic Rare-Event Anomaly Detection with Evidential and Tail-Aware Uncertainty","venue":"ICLR 2026 · Main conference","href":"https://proceedings.iclr.cc/paper_files/paper/2026/hash/0b6566f471cd601cbfc895962623124e-Abstract-Conference.html"},{"title":"Forecasting Solar Flares with the EVEREST Transformer","venue":"TSALM · ICLR 2026 workshop","href":"https://openreview.net/forum?id=KgjNR9TgCB"}],"markdown":"/research/everest-rare-event-forecasting/markdown","summary":"A closer look at rare-event forecasting: what a probability means, how an alert changes a decision, and what the evidence supports.","question":"What makes a rare-event forecast useful enough to act on?","sections":[{"title":"The quiet answer can look excellent","paragraphs":["Imagine reviewing a system that almost always says everything is fine. Most of the time, it is right. Its aggregate accuracy looks reassuring. Then ask a different question: how many of the events that required attention did it catch? A model can perform well on the first question and fail completely on the second.","EVEREST, first-authored by Reific founder Antanas Žilinskas with Robert N. Shorten and Jakub Mareček, was published at the ICLR 2026 main conference. It studies rare-event forecasting from numerical time series. [1]","A related paper by the same authors, “Forecasting Solar Flares with the EVEREST Transformer,” was accepted at TSALM, the ICLR 2026 workshop on Time Series in the Age of Large Models. [7]","The illustration below applies an always-quiet rule to a published evaluation population. It makes the cost of an uninformative answer visible. This rule is an explanatory example, not the research model. The useful question is what we would need to measure before trusting a better answer."],"figure":"rare-events","figureDescription":"M5–72h: 71,729 windows; 71,625 negative, 104 positive. Always-quiet illustration: 99.86% accuracy, 104/104 positive windows missed. Not EVEREST performance; windows are not independent flares. Appendix B, Table 5."},{"title":"What the model is trying to learn","paragraphs":["EVEREST uses a transformer to combine measurements across time and estimate an event probability within a specified horizon. Its training includes objectives for rare cases, uncertainty, and extreme outcomes; inference uses one classification output. Solar flares provide the main evaluation. [2]","The distinction between training and use matters. During training, we can ask more of a model than simply choosing the correct label. We can shape which mistakes receive attention, then examine whether those choices improve the output we actually need. The test remains the observed behavior of that output.","One relevant idea comes from focal loss, developed for object detection: many easy examples can overwhelm the contribution of difficult ones. Reducing their weight gives learning more reason to resolve the remaining errors. That principle concerns the training signal; it does not establish that every unusual observation deserves an alert. [3]","For an operator, the event definition comes first. A brief excursion above a limit and a sustained loss of service are different targets. A prediction about the next hour and a prediction about next week are different commitments. Without those definitions, an apparently precise probability is difficult to test or use."]},{"title":"A probability should earn its meaning","paragraphs":["Calibration asks whether predicted probabilities agree with observed frequencies. Consider a hypothetical collection of comparable forecasts, each assigning an event an 80% probability. If the probabilities are calibrated for that collection, events should occur about four times in five over enough observations. A lone successful prediction cannot establish this property. Guo and colleagues showed why classification performance and probability calibration need separate evaluation. [4]","The practical consequence is that a model's confidence should be checked where it will influence a decision. An average across ordinary conditions may answer a different question from the one posed by the subset that triggers intervention. Sample size also matters: a narrow slice can look reassuring while providing little opportunity to observe failure.","We would therefore want a forecast report to make its reference population visible. Which operating conditions were represented? How often did the event occur? How many examples support the probability range being used? Those details make it possible to distinguish an estimate with evidence behind it from a number that merely looks precise."]},{"title":"Read the errors from both sides","paragraphs":["For ≥M5 flares within 72 hours, Table 12 reports headline means under the paper’s five-seed/bootstrap protocol: TSS 0.966, precision 0.727, and recall 0.966. [2]","These metrics answer different questions. Recall asks how many positive cases were detected. Precision asks how many positive predictions were correct. Saito and Rehmsmeier explain why precision and recall deserve particular attention when positive cases are scarce: an evaluation can otherwise obscure the false alarms within the predicted-positive group. [5]","True Skill Statistic subtracts the false-positive rate from the true-positive rate. It is a discrimination score, not a percentage of predictions that are correct. For a review team, the accompanying counts are often the most useful starting point. Read the detected cases, the missed cases, and the false alarms together, then ask what each would have meant for the person receiving them."]},{"title":"The same forecast can support different alert policies","paragraphs":["A probability threshold connects an estimate to a decision. Changing it changes which observations trigger an alert. If missed events have a different cost from false alarms, a threshold chosen for a balanced metric need not minimize expected cost. Elkan's work on cost-sensitive learning formalizes this separation between estimating probabilities and choosing actions. [6]","The separate diagnostic below makes that choice tangible. Catching the additional positive windows also produces more false-alert windows. The trade-off is visible rather than hidden inside a single score.","In an imagined maintenance workflow, requesting another reading might be inexpensive, while stopping equipment might disrupt a shift. Those actions need not share a threshold. We would first specify the consequence of each action and the evidence required to justify it. We would then evaluate the resulting policy on observations that were not used to choose it.","That is also why a threshold from a research table should stay attached to its task and assumptions. Its value is to reveal how a decision changes, rather than supply a universal setting for another system."],"figure":"alert-thresholds","figureDescription":"M5–72h diagnostic; 104 positive windows. Threshold 0.460: 92 detected, 12 missed, 45 false alerts. Threshold 0.240: 104 detected, 0 missed, 115 false alerts. Miss:false-alarm cost 20:1; separate from headline results. Appendix G."},{"title":"Keep the limits close to the result","paragraphs":["The paper identifies fixed context, data gaps, changing solar cycles, scarce extreme events, and omitted image inputs as limits. Its calibration analysis above probability 0.8 contains only eight windows. [2]","An overall calibration summary cannot settle reliability in a sparsely observed group. Nor can success on one event definition establish performance on another. Our reading is that the useful next question is specific: which new condition would we have to test before allowing this estimate to support that decision?","The same care applies to explanations. A highlighted signal can guide investigation, but deciding that it caused an event requires additional evidence. Keeping prediction, explanation, and action distinct gives each claim a test it can actually pass or fail."]},{"title":"What we carry into Reific","paragraphs":["This research informs how we think about useful temporal evidence. When an agent answers an operational question, we want the answer to preserve what was predicted, the relevant period, the observations supporting it, and the limits of that support. The person reviewing it should be able to inspect the basis for a decision.","The publication is an academic result with its own evaluation scope. Reific's product needs evidence for its own tasks. Connecting those two responsibly means carrying forward the discipline of defining the question, measuring the mistakes that matter, and making uncertainty understandable before an answer becomes an action."]}],"sources":[{"title":"ICLR 2026 proceedings: EVEREST","href":"https://proceedings.iclr.cc/paper_files/paper/2026/hash/0b6566f471cd601cbfc895962623124e-Abstract-Conference.html"},{"title":"EVEREST — final paper and appendices","href":"https://proceedings.iclr.cc/paper_files/paper/2026/file/0b6566f471cd601cbfc895962623124e-Paper-Conference.pdf"},{"title":"Lin et al. (2017): Focal Loss for Dense Object Detection","href":"https://openaccess.thecvf.com/content_iccv_2017/html/Lin_Focal_Loss_for_ICCV_2017_paper.html"},{"title":"Guo et al. (2017): On Calibration of Modern Neural Networks","href":"https://proceedings.mlr.press/v70/guo17a.html"},{"title":"Saito and Rehmsmeier (2015): evaluating precision and recall under imbalance","href":"https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118432"},{"title":"Elkan (2001): The Foundations of Cost-Sensitive Learning","href":"https://cseweb.ucsd.edu/~elkan/rescale.pdf"},{"title":"TSALM 2026 — official accepted-paper list","href":"https://tsalm-workshop.github.io/accepted-papers/#zilinskas2026forecasting"}]},{"slug":"building-demand","title":"Find the half hour that matters.","url":"/research/building-demand","kind":"case-study","date":"2026-09-15","updatedAt":"2026-09-15","author":"Reific","related":[],"publications":[],"markdown":"/research/building-demand/markdown","summary":"A week of building measurements becomes one inspectable energy event, with the original readings and the limits of the conclusion attached.","question":"Which three consecutive 10-minute readings have the highest total appliance energy in the first seven complete days, and what can we conclude?","sections":[{"title":"Start with a question the data can answer.","paragraphs":["We took the first seven complete days from a published building-energy dataset: 1,008 measurements at ten-minute intervals. The question is deliberately narrow: find the three consecutive readings with the greatest combined appliance energy. This gives an agent a concrete event to inspect without asking it to perform arithmetic over every row.","The selected period and method are explicit. The replay enumerates all 1,006 candidate windows, ranks their sums, and preserves the source rows for the maximum. There is no language-model call, learned anomaly detector, or customer deployment behind this example."]},{"title":"The result: 2,390 Wh in three readings.","paragraphs":["The largest sum is 560 + 1,080 + 750 = 2,390 Wh, at the source timestamps 16 January 2016, 18:40, 18:50, and 19:00. Those three ten-minute energy intervals correspond to an average power of 4,780 W. The original CSV lines are 732–734, with the header counted as line 1.","The timestamp labels span twenty minutes; the three energy measurements each describe a ten-minute interval. The source does not say whether a timestamp labels the start or end of that interval, so we preserve its labels rather than inventing an exact interval boundary or timezone."]},{"title":"A maximum is an observation. A fault needs more evidence.","paragraphs":["This window is the maximum in this week. That alone does not make it abnormal, wasteful, or faulty. An aggregate meter cannot identify the appliance responsible. A credible next step is to inspect device meters, schedules, and intervention records before recommending action.","The evidence packet carries that limit alongside the readings. Its usefulness is the audit trail: question, coverage, method, values, source rows, and the information still needed for a stronger conclusion."]},{"title":"Compare against a sensible simple baseline.","paragraphs":["The complete 29-field CSV slice contains 250,068 tokens. Merely selecting the timestamp and appliance-energy columns reduces that to 16,838. The complete evidence JSON contains 529 tokens: 16,309 fewer than the projection, a 96.9% reduction for this specific input payload.","A short SQL query or Python aggregation can compute this result too. This example demonstrates the evidence interface and its exact payload size. It does not establish an advantage over a tool-using agent, better task resolution, or lower total agent cost."]}],"sources":[{"title":"Candanedo (2017), Appliances Energy Prediction — UCI dataset and attribution","href":"https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction"},{"title":"Dataset DOI: 10.24432/C5VC8G","href":"https://doi.org/10.24432/C5VC8G"},{"title":"Creative Commons Attribution 4.0 International","href":"https://creativecommons.org/licenses/by/4.0/"}],"evidence_status":"reference_replay_payload_only"},{"slug":"room-occupancy","title":"A signal is not a diagnosis.","url":"/research/room-occupancy","kind":"case-study","date":"2026-09-15","updatedAt":"2026-09-15","author":"Reific","related":[],"publications":[],"markdown":"/research/room-occupancy/markdown","summary":"A sharp CO₂ increase looks consequential. The separately recorded occupancy labels tell a different story—and show why an evidence packet must preserve uncertainty.","question":"Where is the largest CO2 rise across 30 consecutive sampling intervals, and do those sensor readings alone confirm room occupancy?","sections":[{"title":"Find the change before looking at the answer.","paragraphs":["This replay uses all 8,143 rows from the public dataset's datatraining.txt file. We remove the Occupancy target before building either input payload and before choosing a window. Every sequence of thirty sampling intervals is compared by its endpoint CO₂ change. The timestamps vary between 59 and 61 seconds per interval.","This is a retrospective scan of observed data. No model was trained or tested, and this is not an early-warning result. The source's split name does not turn the replay into a training experiment."]},{"title":"The largest rise is real. Its explanation is unresolved.","paragraphs":["The maximum endpoint rise is 715 ppm, from 650 to 1,365 ppm between 9 February 2015, 21:40:59 and 22:11:00: 1,801 elapsed seconds. The complete 31-reading sequence is preserved in the evidence packet. Its penultimate value is 619 ppm, so the result is dominated by the final jump rather than a steady climb.","A CO₂ rise alone does not establish occupancy, a headcount, or a ventilation fault. The appropriate conclusion from this payload is to flag the change and request corroborating evidence before making that attribution."]},{"title":"The withheld labels expose the inference trap.","paragraphs":["After the window was selected, we inspected the dataset's separate occupancy labels. All 31 are zero: the room is labeled unoccupied throughout this selected window. Those labels were supplied by the dataset authors, whose occupancy record was derived from timestamped images; we did not independently inspect those images.","This is one descriptive counterexample to treating a rise as proof of occupancy. It does not establish a sensor fault or a cause, measure a classifier's accuracy, or prove that any agent would have made the wrong inference. The separate label audit is downloadable and never included in the counted agent-input payloads."]},{"title":"Keep the evidence that can change the answer.","paragraphs":["The compact packet retains every CO₂ reading in the selected window, source rows, elapsed time, and the unresolved conclusion. The full sensor CSV and a simpler two-column projection remain available for comparison. Removing context indiscriminately would undermine the point: the shape of this event is essential to interpreting it.","A code-enabled agent can calculate the same window directly. The question for a future evaluation is whether a proposed evidence layer improves the complete investigation against that capable baseline, including abstentions, errors, review time, and all tool costs."]}],"sources":[{"title":"Candanedo (2016), Occupancy Detection — UCI dataset and attribution","href":"https://archive.ics.uci.edu/dataset/357/occupancy+detection"},{"title":"Dataset DOI: 10.24432/C5X01N","href":"https://doi.org/10.24432/C5X01N"},{"title":"Creative Commons Attribution 4.0 International","href":"https://creativecommons.org/licenses/by/4.0/"}],"evidence_status":"reference_replay_payload_only"}],"representations":{"html":"/","markdown":"/llms-full.txt","json":"/evidence.json","discovery":"/llms.txt"}}