Sentinel-first crop intelligence

1. The resolution-evidence gap

Agricultural parcels are temporal objects. Their spectral signatures evolve through sowing, growth and harvest, while clouds can obscure any single optical acquisition. Sentinel-2 provides the multispectral signal, and Sentinel-1 contributes cloud-independent radar observations. Both are informative, but the labels and core imagery in this experiment live on a 10 m grid.

The central design question is therefore not simply how to upsample. It is how to preserve a scientifically defensible relationship between direct observations and high-grid products. The system separates two claims:

• Coarse-grid supervision: 19-class semantics are labelled on the 10 m PASTIS grid; boundary targets are derived from four-neighbour parcel-instance transitions, and 18 crop-presence targets are derived from the occurrence of semantic classes 1–18.
• Model-derived detail: RGB and logits represented on a 1 m storage grid, constrained to remain consistent with their 10 m anchors.

Relation to prior work

PASTIS [2] established panoptic crop mapping from optical satellite time series with convolutional temporal attention; PASTIS-R [3] extended that setting to paired Sentinel-1 and Sentinel-2 observations. OmniSat [4] studied self-supervised fusion across Earth-observation modalities, while MAESTRO [1] introduced masked autoencoding across multimodal, multitemporal and multispectral inputs. This implementation shares the multimodal motivation but uses a compact supervised temporal-Transformer/FPN architecture. It does not load a pre-trained MAESTRO encoder, and the present experiment does not compare against these systems under a common protocol. The contribution claimed here is therefore an implemented design and evidence audit, not algorithmic novelty or state-of-the-art ranking.

2. Data and supervision

The experiment uses 2,433 complete records from AgriSR-PASTIS, built from PASTIS, PASTIS-R and PASTIS-HD assets. The audited dataset snapshot is Khlaifiabilel/AgriSR-PASTIS revision bd796b56304f57628966f9552afc286ad47f2133; split-manifest and resolved-record hashes are retained in the benchmark report. Records are partitioned according to the standard five-fold split metadata. Folds 1–3 train the model, fold 4 selects checkpoints, and fold 5 forms the internal benchmark.

Radiometric and temporal preprocessing

Sentinel-2 values are divided by 10,000. Sentinel-1 arrays contain VV, VH and a redundant VV−VH channel in decibels; the redundant channel is checked with a 0.25 dB consistency tolerance, then only VV and VH are retained and divided by 20. Training dates are sampled without replacement and sorted chronologically. Validation and benchmark dates are selected deterministically at approximately even temporal intervals.

Let t denote day of year. The temporal coordinate appended to each acquisition uses two seasonal frequencies:

Validity masks reject malformed or absent observations, but they are not cloud masks. The dataset does not provide cloud probability, shadow, snow or saturation layers. Spatial augmentation applies the same random horizontal flip, vertical flip and transpose to every modality and target so cross-sensor registration is not broken.

3. Architecture

Tensor contract

For batch size B and a 10 m crop of height H and width W, the optical tensor has shape [B,T2,10,H,W]. The two radar tensors have shape [B,T1,2,H,W]. The finalised configuration uses feature width C = 96, FPN width C_f = 128, two temporal Transformer layers, four attention heads, feed-forward width 4C and dropout 0.1.

Per-pixel temporal encoding

Each modality begins with a convolutional spectral stem. Instead of forming global space-time tokens, the model reshapes every spatial location into an independent temporal sequence. Self-attention therefore has temporal complexity O(BHW T^2), avoiding the quadratic cost of attention over all THW tokens. A masked temporal mean then produces one feature vector per pixel:

Sentinel-2 has a dedicated encoder. Ascending and descending Sentinel-1 use the same encoder weights, while learned orbit embeddings preserve acquisition-geometry context.

Validity-aware modality fusion

A spatial gate predicts one logit per modality and pixel. Let a_k,h,w ∈ {0,1} indicate whether modality k has at least one valid acquisition at pixel (h,w). When at least one modality is available:

If every modality is invalid at a pixel, the implementation enables only the optical gate to keep the softmax defined; the corresponding all-invalid modality features remain zero. Thus absence is represented explicitly rather than treated as an ordinary zero-valued observation.

The fused representation is concatenated with the optical feature and processed by a feature pyramid at 10 m, 20 m and 40 m, with lateral fusion back to 10 m. This produces three direct heads:

• 19-channel semantic logits at 10 m;
• one-channel parcel-boundary logits at 10 m;
• 18 tile-level crop-presence logits, corresponding to semantic classes 1–18.

Semantic

[B,19,H,W] logits for predicted classes 0–18; target label 19 denotes void and is excluded from prediction.

Boundary

[B,1,H,W] logits derived from four-neighbour parcel-instance transitions.

Crop presence

[B,18] multi-label logits obtained from globally pooled shared features.

Shared feature

[B,128,H,W] FPN representation exposed for optional externally supplied parcel pooling.

High grid

RGB, semantic and boundary tensors with spatial dimensions [10H,10W]; all are model-derived.

Block-consistent detail

The high-grid decoder predicts a residual detail field rather than an unconstrained image. For each non-overlapping 10×10 output block, the residual is projected to zero mean:

Here P averages each 10×10 block and U repeats it. For RGB values and for semantic and boundary logits, P(Y_high) = Y_10m exactly within each model tile before overlap-weighted mosaicking. This identity does not extend through nonlinear softmax or sigmoid transforms: averaged probabilities, argmax labels and majority classes need not recover the 10 m prediction. The constraint preserves the coarse logit anchor while permitting model-derived within-block variation; it does not establish that this variation is physically correct.

Multi-task objective

The optimised scalar objective is a fixed weighted sum:

Semantic cross-entropy ignores void pixels. Soft Dice is averaged only across classes supported in the current batch. Boundary loss combines positive-class-weighted binary cross-entropy and Dice; the positive weight is estimated dynamically and clipped to [1, 50]. Crop presence uses sigmoid BCE.

For supported semantic classes S, probabilities p and one-hot targets y, the Dice component is:

The boundary positive weight is clip(N_negative / max(N_positive,1), 1, 50). Its Dice smoothing constant is 1 rather than 10⁻⁶, making the sparse binary objective numerically defined when few boundary pixels occur. The multiscale SPOT reconstruction applies the Charbonnier penalty ρ(r) = sqrt(r^2 + 10^-6) - 10^-3 after per-image, per-band standardisation.

The degradation term applies a fixed Gaussian blur and average-pooling decimation to the predicted high-grid RGB before a Smooth L1 comparison with the Sentinel-2 RGB anchor. SPOT reconstruction uses per-image, per-band standardisation and Charbonnier penalties at scales 1, 2 and 4, supplemented by horizontal and vertical gradient agreement. SPOT edges supervise only a dilated band around low-resolution parcel boundaries. A differentiable semantic-transition map is finally matched to the high-grid boundary probability.

High-grid semantics have no independent high-resolution semantic target. Their spatial organisation is induced by the coarse semantic anchor, the block constraint, SPOT edge guidance and semantic-boundary consistency.

4. Experimental protocol

The run uses one seed and one fold assignment. Fold 5 has been inspected multiple times and is consequently described as an internal benchmark, not an untouched test set. All four source Sentinel tiles occur in every split, so this protocol does not measure unseen-region generalisation.

Metric definitions

Semantic macro IoU is computed from the accumulated confusion matrix and averages only classes with non-zero union. Foreground macro IoU excludes background class 0. For class c:

Boundary F1 thresholds sigmoid probabilities at 0.5. Precision and recall permit matches within a 3×3 square dilation, corresponding to a one-pixel Chebyshev tolerance on the 10 m grid; this is not one-to-one boundary matching. F1 is computed independently for each 128×128 patch and macro-averaged over 496 patches, so patches rather than boundary pixels are the statistical units. Under the implementation, an empty-truth/empty-prediction patch contributes zero. Crop-presence F1 also uses a 0.5 threshold. Expected calibration error uses 15 confidence bins and is pixel-weighted. RGB PSNR is −10 log10(MSE) under an assumed unit data range; SSIM is a local uniform-window implementation with constants 0.01² and 0.03².

5. Results

The selected checkpoint reached 0.5112 validation macro IoU at epoch 49. The final epoch fell to 0.4915, so the fixed-budget run does not demonstrate convergence.

Class-conditional behaviour

Performance is strongly class dependent, but support alone does not explain the ordering: beet and soybeans substantially outperform winter triticale and sorghum despite supports of the same order of magnitude. Class imbalance, label heterogeneity and spectral or phenological confusion are plausible explanations, not established causes. The low per-class IoUs and crop-presence macro F1 indicate heterogeneous operational risk more clearly than overall pixel accuracy.

The SPOT metrics are diagnostics, not same-date super-resolution scores. Sensor response, atmosphere, illumination, acquisition date and registration all differ. They must not be interpreted as proof of calibrated 1 m optical reconstruction.

In a separate four-patch visualisation diagnostic, model RGB was effectively tied with nearest-neighbour repetition of the Sentinel-2 RGB anchor. Because that comparison covered only four patches and included no full-fold repeat, bicubic baseline or zero-detail baseline, the reported PSNR and SSIM do not establish an improvement attributable to learned detail.

6. Limitations and broader impact

• No repeated trials: one seed, one fold and no confidence intervals prevent variance-aware comparison.
• No geographic holdout: held-out patches share source tiles with training data.
• No causal ablation: the experiment does not isolate the value of Sentinel-1, SPOT guidance or individual losses.
• No high-resolution semantic truth: metre-grid semantic and boundary quality cannot be assigned a true metre-scale IoU.
• Retrospective by default: the full-season sequence may include observations later than a displayed acquisition date.
• Regional taxonomy: PASTIS represents metropolitan France and the French RPG taxonomy; global transfer is untested.

Rare-crop errors can create unequal downstream outcomes. Predictions should not be treated as legal parcel boundaries or used without human and agronomic review for subsidy, insurance, credit or compliance decisions. Every high-grid export should retain provenance identifying it as model-derived.

Falsifiable next experiments

The current run cannot assign causal value to individual design choices. A defensible follow-up should pre-register matched controls and evaluate them with repeated seeds:

7. Reproducibility statement

The v2.0.0 model report records the architecture, data contract, training configuration, metric outputs and artefact hashes. The separate v2.1.0 source-maintenance release retained the same architecture, preprocessing, losses and benchmark record while recording 64 passing tests and 81.42% line coverage; these software-quality results are not additional model-performance evidence.

The model is a new compact temporal-Transformer/FPN implementation. Although the broader NeuralQ project retains attribution and conceptual lineage from MAESTRO, this version does not load the pre-trained MAESTRO masked autoencoder and should not be described as a MAESTRO fine-tune.