Data Ingest
 
Section 6.1: Data Ingest
Note: If you are trying to access files on your server you need to first configure the file access settings. There are lots of ways of getting information and data ingested into RAMADDA. If you have a large and/or dynamic collection of files that you want quick view only access try the Server Side File entry. If you have a directory of data you want fully ingested you can use a harvester.

Server Side File Entry

If you have a very large set of files or you have a set of files that are changing all of the time (e.g,, real-time feeds) it might not be practical or possible to have RAMADDA ingest each and every file. However, you can provide RAMADDA view access to these files by creating a Server Side File Entry.
  • Go to the some destination folder in RAMADDA
  • Select "New Entry" under the File menu.
  • Under the "General" list choose Server Side Files.
  • This will take you to the Server Side File View Entry create form. Enter a name, the server file path and any patterns to use to include or exclude specific files.
  • Once created all of the entries shown under the new entry are dynamic and are not stored in the RAMADDA database.

Data and project organization

You can organize a file system directory tree that can represent project or data sets harvester. There are a number of constructs provided by RAMADDA to help you create and organize your information and data. You might for example have a directory tree as shown below that you want to harvest into RAMADDA. Normally the harvester would create generic File and Folder entries for the directories and files. You can configure the types and metadata with Entry XML Configuration. Copy this dataset.xml as a .ramadda.xml file under the same parent directory before the harvest.
Dataset_1
     datafile1.suff
     datafile2.suff
          ...
Dataset_2
     datafile1.suff
     datafile2.suff
There are numerous other organizational schemes and constructs available. One could have Project information, Sites, Site Visits, etc. The entrtypes and metadata for the data files can be specified as well.