Controlled Parallelism and Resource Policy

by Giux GX-TXT, python-en, scripts-en 2 min read

Scientific Python in GX-TXT · Module 9 · Lesson 54

Parallelism can be useful for independent tasks, but it consumes resources and introduces coordination costs. Before starting worker, we read the GX-TXT policy and maintain a serial path.

Objectives

  • Consult the resource policy.
  • Limit the required workers to the maximum permitted.
  • Use a pool with context manager when allowed.
  • Compare results with a serial calculation.

Execution

Import parallelism.py into the corso_python Session, save it and run it in the Runner.

Script querying the resource policy.
Script querying the resource policy.
from concurrent.futures import ThreadPoolExecutor
from gxtxt import gxtxt_api_call, gxtxt_api_require

gxtxt_api_require("PARALLEL_EXECUTION_API_v1@1")
policy = gxtxt_api_call("PARALLEL_EXECUTION_API_v1", "resources", [])
requested = 2
allowed = max(1, min(requested, int(policy["max_workers"])))
if not int(policy["enabled"]):
    allowed = 1
def square(x):
    return x * x
values = [1, 2, 3, 4]
serial = list(map(square, values))
if allowed > 1:
    with ThreadPoolExecutor(max_workers=allowed) as executor:
        result = list(executor.map(square, values))
    mode = "thread"
else:
    result = list(map(square, values))
    mode = "seriale"
print("Required Worker:", requested, "ammessi:", allowed)
print("Modalita:", mode)
print("Results:", result)
print("Comparison with serial:", result == serial)

The script asks for two workers but limits the number to the maximum reported. If the policy disables parallelism or admits a single worker, it executes the serial branch. With multiple workers admitted, ThreadPoolExecutor uses a context manager that waits for the pool to close. The square function is pure: no change was shared and does not call the Bridge from threads.

Job output showing a single permitted worker and a serial execution path.
Job output showing a single permitted worker and a serial execution path.

In this installation only one worker is allowed: the output shows serial mode, [1, 4, 9, 16] results and agreement with the serial reference. We didn’t measure a speed-up. The work is deliberately small; the overhead of a pool could exceed the benefit.

Threads, processes and results

Threads share memory and require attention to the editable state; processes isolate memory but have startup and transfer costs. The choice depends on the load, runtime and policy, not only on the number of cores. For random calculations it is necessary to assign seeds and flows explicitly and reproducible; for shared writings it is necessary to prevent race. This example is deterministic and does not use random numbers.

Exercise

  1. Why did the script choose the serial branch in the execution shown?
  2. What guarantees the pool context manager over its lifecycle?
  3. A parallel result equal to the serial shows a speed-up?

Worked solution

  1. The policy reported a maximum of a worker.
  2. It manages the closing of the pool at the exit from the block.
  3. No. Demonstrates the agreement for the cases compared; speed requires adequate measurement.

In the final module we will carry out a complete synthetic study.

Download the Module 9 scripts (ZIP)

#Debugging #GX-TXT #Python

Other steps of this project

Leave a Reply

Your email address will not be published. Required fields are marked *