pgScript Developer Guide

Mickael Deloison


Table of Contents

Additional programs required
Flex
Bison
DocBook
Getting the source code
Directory structure
Software architecture
Overview
pgScript architecture
Test pgScript
Unit tests
Integration tests
Preprocessor symbols
NPGADMIN
PGSTEST
PGSDEBUG
Automation scripts
Eclipse project
Add a new expression
Properties of an expression
Operands
New expression
Add a new statement
Properties of a statement
New expression

Additional programs required

Flex

Warning

Flex 2.5.33 or later is required.

You need to use FlexLexer.h provided with your Flex distribution: copy this file into lib/include/pgscript.

For MinGW you can download flex-2.5.33 from MinGW Supplementary Tools download section. regex-0.12 is also required. Unzip the archives in msys/1.0 directory and then:

flex --version

This must print 2.5.33 or later.

Bison

Warning

Bison 2.3 or later is required.

For MinGW you can download bison-2.3 from MinGW Supplementary Tools download section. regex-0.12 is also required. Unzip the archives in msys/1.0 directory and then:

bison --version

This must print 2.3 or later.

DocBook

In order to generate this documentation a DocBook system and the xmlto program are required.

Getting the source code

See INSTALL: From CVS.

Directory structure

  • doc

    Contains files related to the application documentation.

    HTML files are packaged with the application. generate.sh and XML (DocBook) files are used for generating those HTML files.

  • etc

    Contains scripts that are used occasionally.

  • files

    Contains sample input files and test files.

    • inputs

      Sample pgScript scripts.

    • tests

      Integration test suite.

      execute.sh executes this integration test suite. It must be executed from the files/tests folder and pgScript must be compiled. Once executed no warning and no exception should appear.

      • dictionaries

        Sample dictionaries for the dictionary generator.

      • sources

        Input scripts used by execute.sh

  • lib

    Source files and header files for building libpgs.

    parser.sh must be called for regenerating Flex and Bison source files because it does some more processing that just executing bison and flex: it replaces some headers and add other ones.

  • src

    Source files for pgScript main program: this is a simple main() command-line interface that relies on libpgs.

  • test

    Unit test suite.

    Contains a main() in pgsTestSuite.cpp that runs a test suite on libpgs.

    Either src is built or test is built but both cannot be built at the same time: configure takes care of that. Use --enable-tests to compile test and nothing or --disable-tests to compile src. src must be compiled to run the integration test suite.

Software architecture

Overview

This section focuses on the libpgs architecture and therefore the files contained in the lib folder.

  • db

    Source files imported from pgAdmin and used for connecting to a database and executing queries. This code was a bit modified to suit pgScript.

  • include

    Header files of db, pgscript and utils.

  • pgscript

    Source files and header files for building libpgs.

    parser.sh must be called for regenerating Flex and Bison source files because it does some more processing that just executing bison and flex: it replaces some headers and add other ones.

  • utils

    Source files imported from pgAdmin and used by db and log functions. This code was a bit modified to suit pgScript.

When exporting pgScript to pgAdmin it is only necessary to export include/pgscript and pgscript directories because the code of pgScript is fully compatible with pgAdmin without any modification. The opposite is not true and this explains why db and utils code was a bit modified to suit pgScript.

pgScript architecture

This section describes the pgscript folder, which is very like include/pgscript.

  • pgscript

    pgsParser.yy, pgsScanner.ll: Bison & Flex parser for the T-SQL syntax (use parser.sh to generate source and header files).

    parser.tab.cc, lex.pgs.cc: Bison & Flex generated files.

    The parser goes through the to-be-parsed script and builds a statement tree (made of pgsStmt and its children), a statement can be based on an expression (made of pgsExpression and its children). Once the whole script is parsed, the tree is evaluated with the common eval(...) method.

    pgsApplication: interface of libpgs with the outside world, allows users to parse a file or a string.

    • exceptions

      Contains pgScript exceptions which inherits from pgsException.

      pgsArithmeticException: when an operation cannot be performed because of incompatible types for example.

      pgsAssertException: when pgsAssertStmt detects an assertion failure.

      pgsBreakException: when a break instruction is found, must be caught by the enclosing pgsWhileStmt.

      pgsCastException: when a type conversion cannot be performed (string 'abc' to number for example).

      pgsContinueException: when a continue instruction is found, must be caught by the enclosing pgsWhileStmt.

      pgsInterruptException: when the program is stopped as it is running.

      pgsParameterException: when the type of a parameter of a function or a generator is not valid.

    • expressions

      Contains r-values, i.e expressions that return something that can be assigned to a variable. pgsExpression is the common parent class.

      The common interface implies that the children must implement a copy constructor, the assignment operator and a clone() method which does a deep copy and which is based on the copy constructor.

      The eval(pgsVarMap &) method is crucial. During evaluation of the statement tree it performs the operation associated to an expression and returns its values as an r-value (value which can be used in an assignment). It returns a pgsOperand. Its parameter pgsVarMap is a map of pgsOperand.

      pgsOperand is simply a smart pointer on a pgsObject (see objects).

    • generators

      Contains random data generators. pgsObjectGen is the common parent. The wxString random() method returns the random object as a string.

      A lot of generators use a pgsIntegerGen, through a smart pointer.

    • objects

      pgsObject is the common parent class: a smart pointer to a pgsObject (pgsOperand) is the data that is used in pgScript.

      pgsGenerator: container for the generators.

      pgsNumber: represents a number, either a real number or an integer.

      pgsRecord: represents a record set, which is composed of pgsGenerator, pgsNumber and pgsString.

      pgsString: represents a string.

      The eval(...) method returns in general a string representation of the object or the object itself, i.e something that is used in expressions to perform operations.

      Each object must implement some operations. Operations should only be possible between operands of the same type.

      The type of an object is known either by a dynamic cast or by the is_xxx() methods, which use the type specified in the object constructor (pgsTReal, pgsTInt, ...).

    • statements

      Similar to expressions, they perform operation but do not return anything. pgsStmt is the common parent class.

      Those classes do not require copy constructor, assignment operator, clone() method as they are never copied.

      pgsStmtList is a list of statements and therefore owns additional methods: push_xxx().

      pgsProgram is not really a statement: it is the program the root statement that contains the first statement list.

    • utilities

      pgsAlloc: when configured with --enable-debug it tracks each new or delete that occur and at the end of the program execution it shows what new operations have not been deleted.

      pnew, pdelete and pdeletea must be used instead of new, delete and delete[] in order to enable those features.

      pgsContext: utilities for script parsing in pgsParser.yy.

      pgsDriver: manages Bison & Flex files.

      pgsScanner: Flex header.

      pgsMapm: conversion utilities for the MAPM library, the big number library used in pgScript.

      pgsThread: pgScript runs in a detached (auto delete) thread that can be interrupted, this is the thread.

      pgsUtilities: various utilities.

      pgsCopiedPtr, pgsSharedPtr: smart pointers, their names are explicit.

Test pgScript

There are two steps: unit tests on classes and integration tests on pgScript executable.

Unit tests

Go to the pgScript directory and enter the following commands:

./configure --enable-debug --enable-tests
make
./test/pgsTest

There should be no assertion failure, otherwise there is a bug.

Integration tests

Go to the pgScript directory and enter the following commands:

./configure --enable-debug --disable-tests
cd files/tests/
./execute.sh -h 192.168.190.1 -p 5432 -U postgres -W postgres -d testbase

The string -h 192.168.190.1 -p 5432 -U postgres -W postgres -d testbase are the parameters for pgScript. Replace them with your database parameters.

Warning

-d testbase implies pgScript will write to a database called testbase. Be sure it is not a production database.

There should be no error during this process.

Note

Use a LOG_ERRORS log level in main() in order not to display too many things on the screen, otherwise it would be hard to see if there was a failure.

Preprocessor symbols

NPGADMIN

Automatically defined in the standalone version. NPGADMIN stands for Not pgAdmin. This means not to create the set_caller method in pgsApplication. Its goal is to provide pgsApplication with the window and the event to dispatch once a pgsThread is done, in order to notify pgAdmin that pgScript has completed its job and that pgAdmin can disable the stop button and enable the run button again.

When compiled with pgAdmin this symbol is not defined. Therefore set_caller is defined.

PGSTEST

./configure --enable-tests

Compiles tests in the test folder instead of the main program in src folder. Use ./test/pgsTest to run the unit test suite.

PGSDEBUG

./configure --enable-debug

Compiles pgScript (test or src) with memory tracking features: at the end of the program undeallocated memory blocks are displayed. Use pnew instead of new, pdelete(x) instead of delete x and pdeletea(x) instead of delete[] x when programming pgScript in order to be able to use those features.

Automation scripts

autoclean.sh

Cleans every generated file by autogen.sh and object files (*.o). See the source to know the files that are deleted.

Explores ./files, ./lib and ./test. The ./src/Makefile.am is not generated and ./src/configure.ac is not updated with this script.

autogen.sh

Runs aclocal, autoheader, automake and autoconf in ./, ./lib, ./src and ./test.

windows.sh

Generates a Windows binary distribution. pgScript must have been compiled before with MinGW. This puts everything in a pgScript directory that needs to be zipped.

lib/parser.sh

See the Directory structure section.

lib/pgadmin.sh

Generates a tar that can be deployed in pgAdmin. This only contains source files and module.mk files: vcproj file need to be updated if new files are added or some are removed.

doc/generate.sh

See the Directory structure section. Generates documentation.

files/tests/execute.sh

See the Directory structure section. Runs the integration test suite.

Eclipse project

The pgScript project on the CVS repository is an Eclipse CDT project. This is configured for a Windows & MinGW platform.

Project -> Properties -> C/C++ Build -> Environment or Settings

You can configure paths to wxWidgets and libpq in GCC C++ Compiler -> Directories and in C++ Linker -> Libraries. You can use the MINGWDIR environment variable in order to achieve that.

Add a new expression

Properties of an expression

  • Header file is in lib/include/pgscript/expressions, source file is in lib/pgscript/expressions.

  • It returns a value (pgsOperand) once it has been evaluated.

  • It is a pgsExpression child.

  • The header file must be included in pgsExpressions.h.

Operands

A pgsOperand is a smart pointer to a pgsVariable (pgsGenerator, pgsNumber, pgsRecord and pgsString).

To declare a pgsOperand:

pgsOperand operand(pnew pgsNumber(wxT("1")));

Then a pgsOperand is copied, duplicated and deleted automatically. In the example above there is no need to delete the new pgsNumber: the destructor is called automatically when operand is deleted.

pgsOperand copy(operand);
operand = copy;

To duplicate and retrieve the pgsNumber:

pgsNumber * number = operand->clone();

Even if copy and deletion are handled automatically, pgsOperand is accessed like a regular pointer with ->:

cout << operand->value() << endl;
return operand->eval(vars);

New expression

  • Duplicate pgsAssign (header and source) and rename the files: pgsMyExpression for example.

  • Replace each occurrence of pgsAssign with pgsMyExpression.

  • Replace the protected data and the constructor with what you want.

  • Do not forget to modify the destructor, the copy constructor and the assignment operator according to your new data.

  • The value() method must return the not evaluated expression.

  • The eval(vars) method must return the evaluated expression.

Note

The vars parameter is the symbol table: this is a map whose key is the variable name and whose value is a pgsOperand.

  • Add pgsMyExpression.h in pgsExpressions.h.

  • Modify pgscript/pgsScanner.ll (Flex) and pgscript/pgsParser.yy (Bison) to take into account this new expression.

Note

Use parser.sh to regenerate the Flex/Bison parser.

The protected data is most of the time string identifiers and pointers to pgsExpression expressions that are used (evaluated) for evaluating the expression.

Add a new statement

Properties of a statement

  • Header file is in lib/include/pgscript/statements, source file is in lib/pgscript/statements.

  • It returns nothing: this is kind of an expression that returns nothing.

  • It is a pgsStmt child.

  • The header file must be included in pgsStatements.h.

New expression

  • Duplicate pgsPrint (header and source) and rename the files: pgsMyStmt for example.

  • Replace each occurrence of pgsPrint with pgsMyStmt.

  • Replace the protected data and the constructor with what you want.

  • Do not forget to make the copy constructor and the assignment operator private (not allowed).

  • The value() method must return the not evaluated statement.

  • The eval(vars) method evaluate the statement (it returns nothing).

  • Add pgsMyStmt.h in pgsStatements.h.

  • Modify pgscript/pgsScanner.ll (Flex) and pgscript/pgsParser.yy (Bison) to take into account this new statement.