Executions - Packages

How to use the Executions/Packages Framework for script automation and package (execution) triggering.

The Executions/Packages is an event oriented framework that allows enterprise organizations the automation of script processing which can be triggered by:

  • Manually: By clicking the ‘Run’ button on the StreamZero FX Management Server.
  • On Schedule: As a cron job whereas the Cron expression is added on the UI.
  • On Event: Where a package is configured to be triggered bt the FX Router when a specific type of event is observed on the platform.

It allows users to deploy their locally tested scripts without DevOps specific changes or the need to learn complex DSL (description and configuration language). In tandem with Git integrated source code management FX allows distributed and fragmented tech teans to easily deploy and test new versions of code in an agile way with changes being applied immediately.

Contiuous Change Integration / Change Deployment becomes a component based and building block driven approach, where packages can be configurable and parametrised. All scripts and their parameters like secrets and environment variables form packages which makes them reusable for similar jobs or event chains. Event based package triggering allows users to run multiple packages in parallel as a reaction to the same event.

Executions - Packages

Primary entities for “Executions” are packages which are composed by scripts that are executed in a predefined order.

Executions -> Packages

This Use Case defines how to create and run a new package.

  1. Click on Executions on the left side of the dashboard menu to open drop-down
  2. Click on Packages
  3. Click on +Add to create a package

Create Package

Field name Steps & Description
1. Name 1. Name the package
2. Description 2. Descripe the package
3. Project 3. Select the project to which the package will be bound
4. Tags 4. Add Tags of choice manually or select from predefined tags
5. Schedule 5. Schedule cron job -> “Cron like schedule definition. NOTE: day of week start on Monday (0 - Monday, 6 - Sunday) example: “20****” -> The whole definition of Cron Jobs can be found in the next sub-category of this UserGuide
6. Trigger Event Type 6. Select Value -> select event type to trigger the exectution of the package -> please visit the sub-category Events to get a better understanding of how to set event triggers.
7. Allow Manual Triggering 7. Checkbox -> click to allow manual triggering of the package
8. Active 8. Checkbox -> click to set the package to active
9. File Upload (choose file) 9. Click on Choose file (Optional) to upload a script -> upload a JSON “config.json” script to configure the package
10. File Upload (choose file) 10. Click on Choose file (Optional) to upload a script -> upload a python “test_scr.py” script to pull the configuratio from config file and print all items
11. Save 11. Click Save to save packages
Supported File upload Types 4 different file types are supported:
1. “.py file” -> A PY file is a program file or script written in Python, an interpreted object-oriented programming language.
2. “.json file” -> A JSON file is a file that stores simple data structures and objects in JavaScript Object Notation (JSON) format, which is a standard data interchange format.
3. “.sql file” -> A (SQL) file with .sql extension is a Structured Query Language (SQL) file that contains code to work with relational databases.
4. “.csv file” -> A CSV (comma-separated values) file is a text file that has a specific format which allows data to be saved in a table structured format.

config.json script

The config.json file contains key/value configuration pairs that can be accessed in scripts at execution time.

1
{"somekey":"some value 2"}
test_scr.py script

This is an example script that shows how configuration from config.json file can be accessed from a script. package_name will be passed to the script as argument and then can be used for fetching configuration using ApplicationConfigurator from ferris_cli python package.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import sys, json
from ferris_cli.v2 import ApplicationConfigurator

fa = json.loads(sys.argv[1])

package_name = fa['package_name']
config = ApplicationConfigurator.get(package_name)

for k, v in config.items():
    print(f"{k} -> {v}")
    print(v)

Check Created Package

The created package should be triggered every 20 minutes of every hour but can also be run manually.

  • Click on the magnifying glass icon to open the package’s details page

  1. Check details page
  2. Click on “Show Trigger Event”

  1. Check the triggered event details
  2. Close

Package Executions / Runs

  • Click on the “Run” button down the page to run the package manually

It will automatically transfer you to the “List Package Executions” tab

  1. Check runs/package executions to see if you manually triggered execution was processed
  2. Click on the magnifying glass icon of your latest manually triggered run to open details page of the exectuion

  1. Check the details “Show Package Execution” of the run/exection
  2. Click on “List Steps” tab to see the steps of the execution

  1. Check the steps of the run and status (completed; pending; unprocessed; failed)
  2. Click on “Show Results” to verify the script for failed executions

  • Close window

Note that currently only python and sql handlers are available, files of different type will be unprocessed.

Save a Run/Execution

  1. Go back to the “List Package Executions” tab
  2. Click on the edit icon to open make the run/execution editable

  1. Name the execution/run
  2. Describe the execution/run
  3. Click “Saved” check box
  4. Save

  1. Click on Executions to open dropdown
  2. Click on Saved Executions to check the saved run

In the next section “UI Generator”, the importance of the saved run will be showcased.


Last modified September 4, 2023: mobile fix (c3370e3)