FAQ

UNICORE 6

General

What is UNICORE?: UNICORE (Uniform Interface to Computing Resources) offers a ready-to-run Grid system including client and server software. UNICORE makes distributed computing and data resources available in a seamless and secure way through intranets and internet.

Where do I get UNICORE?

You can download UNICORE 6 packages here:

www.unicore.eu/download/unicore6/

Where do I get information about UNICORE?

You can find all the UNICORE 6 documentation at the UNICORE web site:

www.unicore.eu/documentation/

Where do I get help or report bugs?

You can post support requests to the unicore-support mailing list:

unicore-support@lists.sourceforge.net

You can post bug reports and feature request at

UNICORE bug trackers

Alternatively, you can discuss bugs and feature requests on the unicore-devel list (you will have to subscribe)

unicore-devel@lists.sourceforge.net

How is UNICORE 6 different from UNICORE 5?: UNICORE 6 is based on the same principles as UNICORE, but is built on the latest Grid services standards such as WSRF, compliant with the Open Grid Forum's Open Grid Services Architecture (OGSA).

Which version of UNICORE should I use?: As several features of UNICORE 6 are still under development, you may find UNICORE 5 still a better choice in some scenarios. If the current features of UNICORE 6 are sufficient for your needs, or if you are simply interested in using the latest software, use UNICORE 6. We welcome your feedback and suggestions!

Can I use the UNICORE 5 client with UNICORE 6?: No. You will need to use either one of the UNICORE 6 graphical clients or the UNICORE 6 commandline client UCC.

Can I use the UNICORE 6 client with UNICORE 5 servers?: No. The UNICORE 6 clients can not talk to the UNICORE 5 server components. If you have a UNICORE 5 installation on your local resource management system, you can re-use the TSI component with UNICORE 6.

So, how do I start?

Check out the UNICORE Core Server Bundle package on SourceForge.

The package contains a complete UNICORE server installation, including demo certificates and an easy to use graphical installer. You can use the installation as a basis for easily creating your customised installations.

What are the prerequisites for UNICORE?: In order to run the UNICORE 6 server or client components, all you need Sun's Java Runtime Environment (JRE) 1.5 or higher. Since all components are platform independent, they will run under Linux, MAC and windows likewise.

Does UNICORE have a resource management system/batch system?: No. UNICORE is a Grid Middleware, it submits jobs to already installed resource management systems/batch systems.

Do I need a resource management system/batch system?: No. UNICORE can run without a resource management system/batch system, jobs are only forked then. But in production one normally wants a resource management system/batch system.

Is UNICORE 6 compatible with the Globus Toolkit?: Though both UNICORE 6 and the latest Globus Toolkit v4.2 use the same basic technologies (such as WSRF 1.2), they have very different security models, basic services and interfaces, and are thus not directly interoperable.

Certificates

What kind of certificates do I need?

Each server component (Gateway, Registry, UNICORE/X and XUUDB) needs one X.509 private/public key pair signed by a CA (Certification Authority) as identity. Each user needs a signed X.509 private/public key pair which has to be loaded in the keystore of the client. Additionally, you need the certificates of the CAs.

For testing purposes, you can use the same private/public key pair for the user and all the server components. But do so at your own risk!

Where to put the CA (Certification Authority) certificates?

All network connections between the UNICORE components use client-authenticated SSL (Secure Sockets Layer), i.e. both sides of the connection check that they trust the other side. "Trust" means that the CA is checked.

If you use a single CA for all your certificates, the configuration is rather simple: Each server component needs to know the CA certificate, and additionally the CA has to be loaded in the client's keystore.

If you use multiple CA's, consider how the UNICORE components work together: The client communicates with the Gateway, so the Gateway has to know the user's CA and the client has to know the CA of the Gateway. The Gateway also communicates with the Registry and UNICORE/X, so the Registry and UNICORE/X should know the Gateway's CA and vice versa. Additionally, the UNICORE/X communicates with the XUUDB, so both components need to know each other's CA certificates.

Which certificates go into the XUUDB?: When adding a new user to the XUUDB, you need his signed certificate (public key). If you run the XUUDB in DN mode, the distinguished name (DN) of the user's public key will suffice.

How to obtain user certificates?

You can generate a certificate request within the GPE client: Click

Settings -> Keystore Editor

to open up the keystore editor and select

Actions -> Generate Certification Request

The client will create a new private key, which is automatically stored in the keystore editor, and a certification request, which you are asked to save to disk. Send the certification request to a CA (Certification Authority).

You will get a signed certificate (public key) and a CA certificate in return. Store them on disk and click

Action -> Import

to import them into the keystore.

How do I create my own certificates?

Here is a description based on the open source OpenSSL toolkit.

First, you might want to change the default OpenSSL configuration in

/etc/ssl/openssl.cnf

You can store a copy of that file in an arbitrary directory.

Create a x.509 certification authority:
- Generate a private key and a certificate request:
  
  openssl req -config /etc/ssl/openssl.cnf -new -x509 -keyout cakey.pem -out cacert.pem -days 3650
  
  The key will be expire after 365 days.
- Create a file that contains the first serial number (hexadecimal) to be used for signing certificates:
  
  echo "01" > serial
  
  Put the file where openssl finds it (look into openssl.cnf).
- Create an empty index file for the signed certificate database:
  
  touch index.txt
  
  Put the file where openssl finds it (look into openssl.cnf).
- Add cacert.pem as a trusted certificate in the gateways.properties file and njs.properties file.
Sign a certification request:
- Sign a certification request myreq.pem:
  
  openssl ca -config /etc/ssl/openssl.cnf -policy policy_anything -out mycert.pem -in myreq.pem
- Send myreq.pem to the user.
- Extract from myreq.pem the part
```
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
```
  and store it in user.pem
- Add user.pem in the UUDB:
  
  uudb/bin/add user.pem userunixname
Create a user certificate:
- Generate a private key and a certificate request:
  
  openssl req -config /etc/ssl/openssl.cnf -new -keyout mykey.pem -out myreq.pem -days 365
- Send your request myreq.pem to your certification authority.
- Generate a keystore to import in Unicore:
  openssl pkcs12 -export -in mycert.pem -inkey mykey.pem -out mykeystore.p12
- Import mykeystore.p12 in the UNICORE Client:
  Settings->Keystore Editor->Actions->Import Keystore
- Import the certificate of your certification authority cacert.pem in the UNICORE Client:
  Settings->Keystore Editor->Actions->Import Certificate
- Save the UNICORE Keystore:
  File->Save

Grid Layout

How many open ports in the firewall does UNICORE 6 need?: Only the Gateway needs to be accessible from outside the firewall, using one port. The other components need not be accessible outside the firewall. It is possible to bypass the gateway for filetransfers, which will boost performance by a factor of 2, but will need a second open port to allow clients direct connections to the UNICORE/X server.

How many machines do I need for a UNICORE6 system?: You can install UNICORE 6 on a single machine. However, it is recommended to use a dedicated machine for the gateway.

What is the recommended layout for using UNICORE 6 with a cluster system?: We recommend three machines. One hosts the Gateway, one the UNICORE/X server and XUUDB. A third one runs the TSI, usually this will be the login node of your cluster.

Can the UNICORE Gateway serve multiple UNICORE/X or registries?: Yes. If you use static initialisation, enter all VSites in the connections.properties file of the gateway.

Can I share the Registry and XUUDB between multiple UNICORE6 servers?

Yes. The XUUDB is accessed using web service calls, and is configured in the UNICORE server's main configuration file For setting up a shared registry, see this guide.

May I have one UNICORE/X with multiple nodes?: Yes, if you run a batch system like Torque on your nodes. The VSite acts as "front end", so you can access your little cluster through UNICORE.

Then, if I send a job to a VSite, will UNICORE send the job to a free node?: The batch system will do that, yes. It's important to understand that UNICORE does not replace a local batch system.

Must the UNICORE components run as root?: The Gateway, UNICORE/X, and other services should run as normal user. The TSI MUST run as root since it does setuid to the actual user, executes commands, calls the batch system, writes files, etc.

Server Components

How to set up UNICORE 6 server components on Windows?

Since Windows comes without hardly any applications that you can use for job submit, install for example Perl or Python. Then change the IDB (by default unicorex/conf/simpleidb) to recognise that application, here is an example for Python 2.4:


<idb:IDBApplication>
  <idb:ApplicationName>Python Script</idb:ApplicationName>
  <idb:ApplicationVersion>2.4.3</idb:ApplicationVersion>
  <jsdl:POSIXApplication xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
    <jsdl:Executable>C:\Python24\python.exe</jsdl:Executable>
    <jsdl:Argument>-d$DEBUG?</jsdl:Argument>
    <jsdl:Argument>-v$VERBOSE?</jsdl:Argument>
    <jsdl:Argument>$ARGUMENTS?</jsdl:Argument>
    <jsdl:Argument>$SOURCE?</jsdl:Argument>
  </jsdl:POSIXApplication>
</idb:IDBApplication>

It is necessary on windows that you start the server components as administrator.

How can I install UNICORE6 on my cluster running SGE, LoadLeveler...?: The UNICORE 6 quickstart includes documentation on how to access your favorite batch system. See this guide

Clients

What is the meaning of the various job states shown in the client?

UNICORE shows the following job states:

STAGINGIN - the server is staging in data from remote sites into the job directory
READY - job is ready to be started
QUEUED - job is waiting in the batch queue
RUNNING - job is running
STAGINGOUT - execution has finished, and the server is staging out data to remote sites
SUCCESSFUL - all finished, no errors occured
FAILED - errors occured in the execution and/or data staging phases
UNDEFINED - this state formally exists, but is not seen on clients

How to install UCC on Windows?

First, ucc doesn't like if it's installed in a path that does contain any spaces.

The config-directory must be created here: C:\Documents and Settings\<username>\.ucc. The explorer may refuse to create a directory that starts with a dot, in that case you can use mkdir from the command line.