Iterators, Generators and Scientific Streaming
Scientific Python in GX-TXT · Module 9 · Lesson 51
A stream reads data progressively, without requiring to load a whole table at once. A Python generator can deliver a block at a time and keep the reading code separated from processing.
Objectives
- Open a canonical stream and read limited blocks.
- Use
yieldto produce values progressively. - Close the resource with
try/finally. - Distinguish exhaustion of the generator and early interruption.
Execution
At the corso_python Session, import stream_generators.py, save it and run it in the Runner. The example uses four synthetic rows and blocks of two, so the operation is also visible on a small Dataset.

import numpy as np
from gxtxt import gxtxt_api_call, gxtxt_api_require
gxtxt_api_require("TABLE_STREAM_API_v3@3")
api = "TABLE_STREAM_API_v3"
def chunks(dataset_id):
stream = gxtxt_api_call(api, "open_dataset", [dataset_id, ["tempo", "segnale"], 2])
try:
while True:
chunk = gxtxt_api_call(api, "next", [stream])
values = np.asarray(chunk["values"]["segnale"]).reshape(-1)
if values.size:
yield values
if int(chunk["eof"]):
break
finally:
gxtxt_api_call(api, "close", [stream])
print("Stream closed")
item_count = 0
total = 0.0
for chunk in chunks("corso_python_serie_sintetica"):
print("Synthetic block (mV):", chunk)
item_count += len(chunk)
total += float(chunk.sum())
print("Righe:", item_count, "media (mV):", total / item_count)
open_dataset returns a temporary reference to the stream. Each call to next reads the next block; yield delivers the carrier to the external cycle and suspends the function until the next request. The eof signal indicates the end. The finally lock closes the stream when the generator is exhausted or closed.

The output shows two blocks, four rows and an average of 1,4 mV. The processing retains a block at a time; the advantage becomes relevant on larger Dataset. This example does not measure memory consumption and does not claim to demonstrate a specific threshold.
If you stop the cycle before the end, please explicitly close the generator, for example with contextlib.closing(chunks(dataset_id)), to run the finally without depending on Python waste collection. Do not reuse a stream after closing and do not interpret its handle as an arbitrary path of the filesystem.
Exercise
- How many blocks are obtained here and how many values do each contain?
- What is the role of
yield? - Why is it necessary to explicitly close an interrupted generator before the end?
Worked solution
- Two blocks of two values.
- Deliver a value to the caller and suspend the function, keeping the status until the next request.
- To ensure promptly the execution of the
finallyand the release of the stream.
In the next lesson we will use exceptions and logging to make the failures more clear.