EVEREST: when the rare event is the one that matters

The quiet answer can look excellent

Imagine reviewing a system that almost always says everything is fine. Most of the time, it is right. Its aggregate accuracy looks reassuring. Then ask a different question: how many of the events that required attention did it catch? A model can perform well on the first question and fail completely on the second.

EVEREST, first-authored by Reific founder Antanas Žilinskas with Robert N. Shorten and Jakub Mareček, was published at the ICLR 2026 main conference. It studies rare-event forecasting from numerical time series. ¹

A related paper by the same authors, “Forecasting Solar Flares with the EVEREST Transformer,” was accepted at TSALM, the ICLR 2026 workshop on Time Series in the Age of Large Models. ⁷

The illustration below applies an always-quiet rule to a published evaluation population. It makes the cost of an uninformative answer visible. This rule is an explanatory example, not the research model. The useful question is what we would need to measure before trusting a better answer.

What the model is trying to learn

EVEREST uses a transformer to combine measurements across time and estimate an event probability within a specified horizon. Its training includes objectives for rare cases, uncertainty, and extreme outcomes; inference uses one classification output. Solar flares provide the main evaluation. ²

The distinction between training and use matters. During training, we can ask more of a model than simply choosing the correct label. We can shape which mistakes receive attention, then examine whether those choices improve the output we actually need. The test remains the observed behavior of that output.

One relevant idea comes from focal loss, developed for object detection: many easy examples can overwhelm the contribution of difficult ones. Reducing their weight gives learning more reason to resolve the remaining errors. That principle concerns the training signal; it does not establish that every unusual observation deserves an alert. ³

For an operator, the event definition comes first. A brief excursion above a limit and a sustained loss of service are different targets. A prediction about the next hour and a prediction about next week are different commitments. Without those definitions, an apparently precise probability is difficult to test or use.

A probability should earn its meaning

Calibration asks whether predicted probabilities agree with observed frequencies. Consider a hypothetical collection of comparable forecasts, each assigning an event an 80% probability. If the probabilities are calibrated for that collection, events should occur about four times in five over enough observations. A lone successful prediction cannot establish this property. Guo and colleagues showed why classification performance and probability calibration need separate evaluation. ⁴

The practical consequence is that a model's confidence should be checked where it will influence a decision. An average across ordinary conditions may answer a different question from the one posed by the subset that triggers intervention. Sample size also matters: a narrow slice can look reassuring while providing little opportunity to observe failure.

We would therefore want a forecast report to make its reference population visible. Which operating conditions were represented? How often did the event occur? How many examples support the probability range being used? Those details make it possible to distinguish an estimate with evidence behind it from a number that merely looks precise.

Read the errors from both sides

For ≥M5 flares within 72 hours, Table 12 reports headline means under the paper’s five-seed/bootstrap protocol: TSS 0.966, precision 0.727, and recall 0.966. ²

These metrics answer different questions. Recall asks how many positive cases were detected. Precision asks how many positive predictions were correct. Saito and Rehmsmeier explain why precision and recall deserve particular attention when positive cases are scarce: an evaluation can otherwise obscure the false alarms within the predicted-positive group. ⁵

True Skill Statistic subtracts the false-positive rate from the true-positive rate. It is a discrimination score, not a percentage of predictions that are correct. For a review team, the accompanying counts are often the most useful starting point. Read the detected cases, the missed cases, and the false alarms together, then ask what each would have meant for the person receiving them.

The same forecast can support different alert policies

A probability threshold connects an estimate to a decision. Changing it changes which observations trigger an alert. If missed events have a different cost from false alarms, a threshold chosen for a balanced metric need not minimize expected cost. Elkan's work on cost-sensitive learning formalizes this separation between estimating probabilities and choosing actions. ⁶

The separate diagnostic below makes that choice tangible. Catching the additional positive windows also produces more false-alert windows. The trade-off is visible rather than hidden inside a single score.

In an imagined maintenance workflow, requesting another reading might be inexpensive, while stopping equipment might disrupt a shift. Those actions need not share a threshold. We would first specify the consequence of each action and the evidence required to justify it. We would then evaluate the resulting policy on observations that were not used to choose it.

That is also why a threshold from a research table should stay attached to its task and assumptions. Its value is to reveal how a decision changes, rather than supply a universal setting for another system.

Keep the limits close to the result

The paper identifies fixed context, data gaps, changing solar cycles, scarce extreme events, and omitted image inputs as limits. Its calibration analysis above probability 0.8 contains only eight windows. ²

An overall calibration summary cannot settle reliability in a sparsely observed group. Nor can success on one event definition establish performance on another. Our reading is that the useful next question is specific: which new condition would we have to test before allowing this estimate to support that decision?

The same care applies to explanations. A highlighted signal can guide investigation, but deciding that it caused an event requires additional evidence. Keeping prediction, explanation, and action distinct gives each claim a test it can actually pass or fail.

What we carry into Reific

This research informs how we think about useful temporal evidence. When an agent answers an operational question, we want the answer to preserve what was predicted, the relevant period, the observations supporting it, and the limits of that support. The person reviewing it should be able to inspect the basis for a decision.

The publication is an academic result with its own evaluation scope. Reific's product needs evidence for its own tasks. Connecting those two responsibly means carrying forward the discipline of defining the question, measuring the mistakes that matter, and making uncertainty understandable before an answer becomes an action.