Reproducible Preprocessing and Numerical Methods
Scientific Python in GX-TXT · Module 10 · Lesson 57
Before the fit we prepare a view of the data necessary for analysis and check that the values and uncertainties are usable. We keep the synthetic source separated from the derived output.
Objectives
- Project variables into a new Dataset.
- Check finiteness and positivity of uncertainties.
- Interpolate some points without treating them as measurements.
- Document the choices without deleting lines to improve the fit.
Execution
After lesson 56, import preprocessing.py to the corso_python Session, save it and execute it. The transformation creates corso_finale_a_selezionato: if the ID exists, choose a free ID and update it even in subsequent readings.

import numpy as np
from gxtxt import gxtxt_api_call, gxtxt_api_require
for api in ("DATASET_TRANSFORM_API_v1@1", "CANONICAL_DATASET_API_v2@2", "NUMERIC_API_v1@1"):
gxtxt_api_require(api)
source = "corso_finale_a"
output = "corso_finale_a_selezionato"
manifest = {"api_version": 1, "operation": "project_variables",
"sources": [{"dataset_id": source, "variable_ids": ["tempo", "segnale", "sigma"]}],
"output": {"dataset_id": output, "contract": "auto", "storage": "auto"},
"options": {}, "schema_policy": {}, "metadata_policy": "inherit_single_source"}
result = gxtxt_api_call("DATASET_TRANSFORM_API_v1", "execute", [manifest])
r = gxtxt_api_call("CANONICAL_DATASET_API_v2", "read_range", [output, ["tempo", "segnale", "sigma"], 0, 21])
t = np.asarray(r["tempo"]["values"]).reshape(-1)
y = np.asarray(r["segnale"]["values"]).reshape(-1)
sigma = np.asarray(r["sigma"]["values"]).reshape(-1)
valid_values = np.isfinite(t) & np.isfinite(y) & np.isfinite(sigma) & (sigma > 0)
query = np.array([0.125, 1.125, 2.125])
estimates = gxtxt_api_call("NUMERIC_API_v1", "interpolate", [t, y, query, {"method": "linear", "extrapolation": "error"}])
print("Source:", source, "output:", result["dataset_id"])
print("Issues valid:", int(valid_values.sum()), "su", len(t))
print("Time interpolated (s):", query)
print("estimates, non-measures (mV):", np.asarray(estimates["yq"]).reshape(-1))
print("No line excluded from the fit")
project_variables publishes a derivative copy with time_values, signal and sigma, keeping the source separate with the ideal curve. The mask controls that time, signal and uncertainty are finite and that sigma is positive. In the result shown all 21 rows pass the checks; the mask does not delete rows.

The call interpolate calculates three linear estimates between the original times. They are derived values, not new observations; they are not added to the points of the fit of the next lesson. The extrapolation: error option rejects requests outside the domain. If a check fails, it would be necessary to document and motivate the management of the row, not to remove it only because it worsens the result.
Exercise
- What Dataset remains the original source?
- Are interpolated estimates new measurements?
- Why check that
sigmais positive?
Worked solution
corso_finale_a; output has a distinct identity.- No, they’re estimates according to the linear method.
- A standard uncertainty used for inverse weights must be positive and finite.