Shared directories

You can register csv data and udb data stored locally or shared in network to iServer to use the csv files, udb datasets and subfolders in distributed analysis.

Register a file share

Steps:

1. On the iServer service management page (http://supermapiserver:8090/iserver/manager), click on "Cluster">"Data Registration" to enter data registration page;
2. Click "Register data storage" to register a data store;
3. Input "Storage ID", select "Big data file share" in "Data storage type" dropdown arrow, and choose "Share directory" in "Shared data type" dropdown arrow;
4. When configuring the "Shared file directory", you can do the following operations:

According to the number of Spark cluster nodes you want to use:

1. • • If there is only one worker node in the Spark cluster, the data can be directly placed in a file path on the machine where the worker node locates, for example: /home/supermap/data
     • If there are multiple Spark Worker nodes, first you need to set up a network share for the directory where the data resides, then map the shared directory to the local disk of the worker node. Note, you need to map for each computer, and the mapped disk path for each computer requires being consistent with each other. After these preparation work, as the administrator, you should input the mapped directory in "Shared file directory" when registering a file directory.

Recommendation: When there are multiple Spark Worker nodes, to avoid mapping drive conflicts, don't use the computer where the data resides as a Spark Worker node.

According to the file type to be registered:

1. • • If you're registering a udb or csv file, you should directly fill in the directory path where the file is located, for example: /home/supermap/data
     • If you are registering a file directory with csv files, you need to fill in the parent directory of the directory where the csv files locate. For example, you should fill "Shared file directory" with "F:\file\test" in the picture below. And the format of the fields and attributes of csv files must be the same. (Only supports open the csv file directory in read-only way)

5. Click "Register data Storage" button to complete the registration.

       Now you can start to perform distributed analysis services.

       Note: If you are registering csv files, the data needs to be verified before it can be used for distributed analysis service. For details, see: csv data verification.