Enable Strict Security on FileCatalyst Applications with Strict Security
Posted by Aly Essa, Last modified by Aly Essa on 18 June 2020 11:17 AM

Overview

The FileCatalyst Server uses a self-signed certificate (by default) to encrypt communications over the Secure Server Port. This port is used to authenticate users to the FileCatalyst Server and pass messages from the Server to various client-side FileCatalyst applications. The self-signed certificate can be replaced by one obtained from a Trusted Certificate Authority, such as Thawte and GoDaddy or you can import your own self-signed certificate. 

The Strict SSL feature in the FileCatalyst Server, enables a client-side application (HotFolder, TransferAgent, CLI) to reject the FileCatalyst Server’s SSL security certificate if it is not signed by a Trusted CA. However, you can add certificates to be trusted that were either not obtained from a trusted certificate vendor and are self-signed as well. This article will explore a couple of different ways to install a SSL Certificate into a FileCatalyst Server. Use the following guide to enable Strict SSL Security on client-side FileCatalyst applications:
http://support.filecatalyst.com/index.php?/Knowledgebase/Article/View/360/0/enabling-strict-security


If you are using FileCatalyst Server v3.5 or older you can also use the method to install a SSL Certificate outlined in this article:
http://support.filecatalyst.com/index.php?/Knowledgebase/Article/View/293/0/how-to-install-a-ssl-certificate-into-a-filecatalyst-server

Requirements :

Environment

FileCatalyst Server v3.7 and newer.

Resolution

The following steps have been written while testing this method in Linux OS however, the commands do translate over to Windows Environment as well. This section will outline different methods of deploying a SSL Certificate into the FileCatalyst Server.

Section A.
Java Keystore Issued from a Trusted Certificate Authority

This method requires the following:

  • The keystore file provided by your vendor and it will use the following naming convention: mybusinessname.keystore
  • The keystore password used by the vendor for your certificate request. For this method, we will use the password kspwd.
  • The alias used by the vendor for your server certificate at the bottom of the certificate chain. For example, myserver.
  1. Convert the vendor-provided Java Keystore (JKS) file to a PKCS12 format keystore. Choose a new file name and password for the output keystore file.

    Command:

    keytool -importkeystore -srckeystore mybusinessname.keystore -destkeystore mybusinessname.p12 -deststoretype PKCS12 -srcalias myserver -deststorepass kspwd -destkeypass kspwd

    Console Output:

    Enter source keystore password: kspwd

  2. Extract the public certificate chain from the PKCS12 keystore. Choose a new file name for the output certificate.

    Command:

    openssl pkcs12 -in mybusinessname.p12 -nokeys -out servercert.pem

    Console Output:

    Enter import password:  newpwd

  3. Extract the private key from the PKCS12 keystore. Choose a new file name and password for the output key file.

    Command:

    openssl pkcs12 -in mybusinessname.p12 -nocerts -out serverkey.pem

    Console Output:

    Enter import password: newpwd

    Enter PEM pass phrase: secret

    Verifying - Enter PEM pass phrase: secret

  4. Convert the PKCS12 key file to RSA. Choose a new file name and password for the output key file.

    Command:
     

    openssl rsa -in serverkey.pem -outform PEM -DES3 -out serverkey.pvk 

    Command:

    Enter pass phrase for serverkey.pem: secret

    Enter PEM pass phrase: finalpwd

    Verifying - Enter PEM pass phrase: finalpwd

  5. Open the FileCatalyst Server Remote Admin and click on the Security tab. Edit the Configure SSL Port section, with the paths to servercert.pem and serverkey.pvk, and the password created in step 4. Hit Apply to accept the changes.

Section B. 
Creating a Self-Signed Certificate and Private Key


  1. Create a certificate and private key pair. Follow prompts to choose a new password and enter the information for your organization.

    Command:

    openssl req -x509 -newkey rsa:2048 -keyout serverkey.pem -out servercert.pem -days 365

    Console Output:

    Enter PEM pass phrase: secret

    Verifying - Enter PEM pass phrase: secret

    You are about to be asked to enter information that will be incorporated into your certificate request. Enter the information in the appropriate fields.

  2. Convert the private key.

    Command:

    openssl rsa -in serverkey.pem -outform PEM -des3 -out serverkey.pvk

    Console Output:


    Enter pass phrase for serverkey.pem: secret

    Enter PEM pass phrase: finalpwd

    Verifying - Enter PEM pass phrase: finalpwd

  3. Open the FileCatalyst Server Remote Admin and click on the Security tab. Edit the Configure SSL Port section, with the paths to servercert.pem and serverkey.pvk, and the password created in step 2. Hit Apply to accept the changes.

  4. Since a self-signed certificate will not be recognized by clients using the default TrustStore, you must create a new TrustStore file containing the new certificate, or insert the certificate into an existing TrustStore file. If you want clients to only recognize this certificate, go to section C. If you want clients to recognize this certificate in addition to ones issued by the default trusted Certificate Authorities (CAs), go to section D.

Section C:

Creating a New TrustStore for the Certificate

  1. Create a new Java Keystore (JKS) TrustStore using the servercert.pem certificate file. Choose a new alias, file name, and password.

    Command:
     

    keytool -import -trustcacerts -alias selfsigned -file servercert.pem -keystore selfsigned.jks

    Console Output:

    Enter keystore password: changeit

    Re-enter new password: changeit

    Trust this certificate? [no]:  yes

  2. Configure the HotFolder Client to use the new TrustStore instead of the default JRE TrustStore by configuring a new system property:

    From the HotFolder Administration application go to Setting tab and under System Properties click  Add Property. Enter the following properties: 

    Property Name: system.property.unlimited.fc.deployment.security.enforcement.truststore
    Value: /path/to/selfsigned.jks

    Hit Apply to save your changes.

  3. Restart the FileCatalyst Application and Service.

Section D:
Adding the Certificate to the Existing TrustStore

  1. Copy the cacerts file found in the JRE to the directory where the certificate is located. 

    The cacerts file is usually found in /path/to/java/jre/lib/security/cacerts. Rename the file from cacerts to cacerts.jks. Please note that the original cacerts file has no extension.

  2. Import the certificate into the cacerts copy.

    Command:

    keytool -import -trustcacerts -alias selfsigned -file servercert.pem -keystore cacerts.jks

    Console Output:

    Enter keystore password: changeit

    Trust this certificate? [no]:  yes

  3. Configure the HotFolder Client to use the new TrustStore instead of the default JRE TrustStore by configuring a new system property:

    From the HotFolder Administration application go to Settings tab and under System Properties click  Add Property. Enter the following: 

    Property Name: system.property.unlimited.fc.deployment.security.enforcement.truststore
    Value: /path/to/cacerts.jks

    Hit Apply to save your changes.

  4. Restart the FileCatalyst Application and Service.




Comments (0)