Reducing build time / memory requirements for AgML compilation

The main problems during AgML compilation are (1) the time it takes to compile the STAR geometry (approximately 12 minutes), and (2) the amount of memory required to compile the code (approximately 4GB or more).   This problem is caused primarily by the reliance on the ROOT dictionary, and the creation of the dictionary for approximately 5000 user classes.  The ROOT dictionary is used for two purposes.  (1) The implementation of a factory pattern for the creation of user volumes, and (2) the implementation of the AgML structures analagous to the Structures defined in the AgSTAR language.  Of the two usages, #1 causes the largest share of the problem, as 95% of all classes in the ROOT dictionary are used-defined volumes.

Reducing the compile time could be achieved through one of several paths.  They are:

  1. Modify the build scheme so that one dictionary (_Cint.cxx file) is created for each geometry module
  2. Modify the build scheme so that one library is built for each detector subsystem
  3. Modify the AgML support library/export codes so that the ROOT dictionary is not needed to construct geometry volumes... i.e. implement a AgBlockFactory
    • Would reduce the overall number of classes required in the ROOT dictionary
    • AgStructure remains the same implementation

 

Quick and Dirty Test #1

Victor reminded me that we can wrap parts of classes with the #ifndef __CINT__ ... #endif cpp directives, so that we do not generate dictionary entries for every method defined in a class.  The classes which define blocks are fairly lightweight in AgML, and there were only 2 methods which needed to be wrapped per class.

Compilation time: 10 minutes... no significant change.

Quick and Dirty Test #2

Just to double check my understanding, I also wrapped most of the methods in the AgBlock base class so that they wouldn't be defined by CINT.

Compilation time: 10 minutes.... again, nothing significant.

Quick and Dirty Test #3

This involves some minor surgery.  Now, for every Block defined by the user, AgML inserts a static method defined as

static AgBlock *New(){ return new BBCM(); }

... or whichever block the user has defined in XML.  We can then eliminate the ROOT dictionary for all of the user classes, by changing the code AgModule...

AgModule::AddBlock( const Char_t *name ) 

becomes

AgModule::AgBlock( const Char_t *name, AgBlock* (*ctor)() )

... i.e. we pass in the "New" method defined by the block, as well as the name of the block.

Compilation time: 3 minutes.  This is significant.

 After talking with Victor, ... realized that this factory creation pattern can be replaced with a singleton pattern.

Conclusion:  We can cut compilation time significantly by eliminating the dictionary entries for all of the user defined blocks in the geometry.


Implementation:

The only places where the ROOT dictionary is used are (1) in the creation of the AgBlock-derived objects (which correspond to a developer's <Volume ...> ... </Volume> declarations), and (2) the manipulation of the AgStructure objects.  The former is straightforward to factor out.  Only one instance of each of the user's AgBlock per <Volume> declaration exists in memory at any one time.  i.e. there is only one ECAL block, only one TPCE block, only one SVTT block etc...  So.  The user's AgBlock objects can be implemented as singleton classes.  This eliminates the need to create them via a factory pattern, as is currently done.  We replace

AddBlock("BBCM")

with

AdddBlock("BBCM", BBCM::Instance() )

in the BbcmGeo.cxx file for the bbc module. 

Validation:

I create a CINT file for the geometry compiled in DEV.  I compare it line-by-line with a CINT file using my modifications.  I see all lines in the two files are the same for teh y2006, y2007, y2008,y2009, y2010, y2011 and y2012 geometry tags.  I conclude that this refactor of the code has no impact on the geometry produced in AgML.

 

-----------------------------------------------------
y2006
-----------------------------------------------------
nlines: 83599
nsame:  83599
-----------------------------------------------------
y2006h
-----------------------------------------------------
nlines: 98127
nsame:  98127
-----------------------------------------------------
y2007
-----------------------------------------------------
nlines: 93540
nsame:  93540
-----------------------------------------------------
y2007h
-----------------------------------------------------
nlines: 89424
nsame:  89424
-----------------------------------------------------
y2008
-----------------------------------------------------
nlines: 78356
nsame:  78356
-----------------------------------------------------
y2008e
-----------------------------------------------------
nlines: 95758
nsame:  95758
-----------------------------------------------------
y2009
-----------------------------------------------------
nlines: 73657
nsame:  73657
-----------------------------------------------------
y2009d
-----------------------------------------------------
nlines: 95726
nsame:  95726
-----------------------------------------------------
y2010
-----------------------------------------------------
nlines: 99723
nsame:  99723
-----------------------------------------------------
y2010c
-----------------------------------------------------
nlines: 99783
nsame:  99783
-----------------------------------------------------
y2011
-----------------------------------------------------
nlines: 99560
nsame:  99560
-----------------------------------------------------
y2011a
-----------------------------------------------------
nlines: 99560
nsame:  99560
-----------------------------------------------------
y2012
-----------------------------------------------------
nlines: 95167
nsame:  95167
-----------------------------------------------------
dev13
-----------------------------------------------------
nlines: 95178
nsame:  95178