Database Building

A ‘build’ of a mine is a complete data loading run starting from an empty database. It is recommended that you use the project_build script. The build script runs the data integration and any post-processing steps.

Each mine has an integrate project that reads the project.xml file and builds the data warehouse. This steps through each source defined in the project.xml file and transates the specified data from a source format and loads it into the production database. Data integration is governed by primary keys, any conflicts are resolved by a priorities config file.

• project_build script
  • Command line options
  • Running a Single Datasource
  • Running a Custom Datasource
• Project XML
  • <sources>
  • <post-processing>
  • Versions
  • Data model
  • Examples
• Data Integration
• Model Merging
  • Configuration
  • Example
  • Global Additions File
• Primary Keys
  • Data source keys configuration files
  • Global primary key configuration file [DEPRECATED]
    • Define keys in a central file [DEPRECATED]
    • Using keys (from central file) in each source [DEPRECATED]
• Priority Configuration
  • File format
  • Class Hierarchy
  • Validation
• Post processing
  • Sequence Features
    • create-chromosome-locations-and-lengths
    • transfer-sequences
    • create-references
    • create-utr-references
    • create-intron-features
    • make-spanning-locations
  • Overlapping and Flanking Features
    • create-intergenic-region-features
    • create-location-overlap-index
    • create-bioseg-location-index
    • create-overlap-view
    • create-gene-flanking-features
  • Data
    • do-sources
  • Webapp
    • create-attribute-indexes
    • create-search-index
    • populate-child-features
    • summarise-objectstore
    • create-autocomplete-index
• Post build updating with SQL triggers
  • Requirements
  • Procedure
    • Adding triggers
    • Removing triggers
    • What can’t be done (yet)
• Debugging
  • Given a ProxyReference, but id not in ID Map
    • Error message
    • Problem
    • Solution