datconv.outconn package

Common data or structures to be used in Datconv Output Connectors.

datconv.outconn.STRING = 1

Bit meeaning that Connector accepts strings.

datconv.outconn.OBJECT = 2

Bit meeaning that Connector accepts objects.

datconv.outconn.LIST = 4

Bit meeaning that Connector accepts list objects.

datconv.outconn.ITERABLE = 8

Bit meeaning that Connector accepts Python iterable objects.

Output Connector interface

This module contain skeleton class suitable as starting point for new Datconv output connectors.

class datconv.outconn._skeleton.DCConnector[source]

Bases: object

This class must be named exactly DCConnector. It is being called by Writter in order to write output data. It’s main task is to:

  • write data to destination storage (which can be file, database, some stream)

Additional constructor parameters may be added to this method, but they all have to be named parameters. Parameters are usually passed from YAML file as subkeys of OutConnector:CArg key.

supportedInterfases()[source]

Obligatory method that must be defined in Output Connector class. Informs Writer about kind of interface this connector implements. Connector should return one or combination of flags: STRING, OBJECT, LIST, ITERABLE. If is called by Writer before it pass any data to connector.

pushString(strData)[source]

Obligatory method that must be defined if Connector returned STRING flag in supportedInterfases() method. It is called by Writer to pass data to be written to output. Connector must write passed data to output. Method does not return any value.

Parameters:strData – string to be written to output.
getStreams()[source]

Obligatory method that must be defined if Connector returned STRING flag in supportedInterfases() method. It may be called by Writer to obtain output stream to write to (instead of calling pushString. It may happen if Writer uses some library functions that require stream to be passed to. Connector must return array of its output streams (must be a list - typically one element).

Returns:list of streams for writing data.
tryObject(obj)[source]

Obligatory method that must be defined if Connector returned OBJECT flag in supportedInterfases() method. It is called by Writer before it pass any data to connector (but after supportedInterfases) to check if it is configured with compatible connector. Passed object is of the same type like it will be later passed to pushObject() method. Connector should call isinstance() and return True if object is of right type.

Parameters:obj – object passed for hands shaking.
Returns:boolean value indicating if passed object is of correct type.
pushObject(obj)[source]

Obligatory method that must be defined if Connector returned OBJECT flag in supportedInterfases() method. It is called by Writer to pass data to be written to output. Connector must write passed data to output. Method does not return any value.

Parameters:obj – object to be written to output.
onFinish(bSuccess)[source]

Optional method that may be defined. If it is defined it is called by Writer at and of conversion process. Connector may use it to close its streams, commit data or just log proper message. Method does not return any value.

Parameters:bSuccess – parameter that inform connector if process is going to end sucessfully.

datconv.outconn.dcnull module

This module implements Datconv Output Connector which discards output (like writing to /dev/null).

datconv.outconn.dcstdout module

This module implements Datconv Output Connector which writes data to standard output stream.

datconv.outconn.dcfile module

This module implements Datconv Output Connector which saves data to regular file.

class datconv.outconn.dcfile.DCConnector(path, mode='w')[source]

Bases: object

Please see constructor description for more details.

Parameters are usually passed from YAML file as subkeys of OutConnector:CArg key.

Parameters:
  • path – relative or absolute path to output file.
  • mode – output file opening mode.

For more detailed descriptions see conf_template.yaml file in this module folder.

datconv.outconn.dcexcel module

This module implements Datconv Output Connector which writes data to MS Excel (*.xlsx) file. This connector should be used with Writer: datconv.writers.dccsv module. It requires Python package openpyxl to be installed. It does not require Excel program to be installed.

Note

This is very initial version of the package (beta quality):

  • there is no support for cell value types: currently all data are placed as text
  • output uses default font, sheet name etc.
  • do not use this connector with very large data: all data are kept in memory before baing saved.
class datconv.outconn.dcexcel.DCConnector(path)[source]

Bases: object

Please see constructor description for more details.

Parameters are usually passed from YAML file as subkeys of OutConnector:CArg key.

Parameters:path – relative or absolute path to output file.

For more detailed descriptions see conf_template.yaml file in this module folder.

datconv.outconn.dcmultiplicator module

This module implements Datconv Output Connector which sends data to multiply connectors.

class datconv.outconn.dcmultiplicator.DCConnector(clist=[], ilist=[])[source]

Bases: object

Please see constructor description for more details.

Parameters are usually passed from YAML file as subkeys of OutConnector:CArg key.

Parameters:
  • clist – list of sub-connectiors that will get output records. Every item should be a dictionary with Module: key and optional CArg: key.
  • ilist – list of sub-connectiors instances that will get output records. Every item should be a created instance of other output connector - option to use only in code.

Items passed in clist parameter will be instantiated by constructor, items passed in ilist parameter are already live instancies that will be added to sub-connectors list. For more detailed descriptions see conf_template.yaml file in this module folder.

Configuration keys

Listing of all possible configuration keys to be used with output connectors contained in this package.

There are sample values given, if key is not specified in configuration file, than default value is assumed.

OutConnector:
    Module: datconv.outconn.dcnull

OutConnector:
    Module: datconv.outconn.dcstdout

OutConnector:
    Module: datconv.outconn.dcfile
    CArg:
        # relative or absolute path to output file; obligatory
        path: "out/AcctAgentAdjustment_c5019_s38.xml"
        
        # output file opening mode (w or a); optional
        # default: w
        mode: a

OutConnector:
    Module: datconv.outconn.dcexcel
    CArg:
        # relative or absolute path to output file; obligatory
        path: "out/AcctAgentAdjustment_c5019_s38.xlsx"
             
OutConnector:
    Module: datconv.outconn.dcmultiplicator
    CArg:
        # list of sub-connectiors that will get output records;
        # every sub-connection may support different interface.
        # default: []
        clist:
            - Module: datconv.outconn.dcfile
              CArg:
                path: "out/AddnDrawNbrs_c5019_s38_1.json"
            - Module: datconv.outconn.crate.json
              CArg:
                path: "out/AddnDrawNbrs_c5019_s38_2.json"
                cast:
                    - [['rec', 'value'], str]
                lowercase: True