Knowledgebase : Pre-Sales

Overview

All of our FileCatalyst Products are installed with default memory limits, including both initial reserve and maximum memory. However, these can be configured to fit your specific needs and environment. This article will go through the configuration process for FileCatalyst Workflow or Webmail for Windows, Linux, and MacOSX

NOTE:

  • For all environments:
    If you are using a 32-bit Operating System or Java 32-bit, the maximum memory you can define for JAVA is limited to 1.5GB of memory.  64-bit Operating Systems running 64-bit Java do not have such memory limitations.
  • It is also recommended (but not required) to set the minimum and the maximum memory to the same value, as this allows JAVA to allocate a single continuous block of the memory from the OS on startup.
  • Have all Tomcat Server Service shut down before editing any configuration files.


Environment

FileCatalyst Workflow v4.9 and later.

FileCatalyst Webmail v4.9 and later.

 

Resolution 

Windows

When FileCatalyst Webmail or Workflow is installed on a Windows machine, there are two ways to increase the memory used.

  1. The first is to use the tomcat7w.exe or tomcat8w.exe, which is located in the \apache-tomcat\bin folder. A sample path would be C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin\.
    1. Run tomcat7w.exe or tomcat8w.exe as an administrator. You can right-click the file and select the Run as Administrator.
    2. When the Tomcat Properties opens, select the Java tab.
    3. Increase Initial and Maximum Memory Pools as required. We recommend using identical values so the memory block is contiguous.

  2. The second method is to edit your catalina.bat file.
    1. Open the catalina.bat file in a text editor, which is located in the \apache-tomcat\bin. A sample path would be C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin\
    2. Add the following line:

      set JAVA_OPTS="-Xms1024m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=356m -XX:PermSize=256m -XX:MaxPermSize=356m"

      Again we recommend using identical values for both Xms and Xmx so your memory block is contiguous.

Linux

For a deployment of FileCatalyst Webmail or Workflow is installed on a Linux machine, the catalina.sh startup script needs to be editted.

  1. Navigate to the /apache-tomcat/bin/ folder and open the catalina.sh file in a text editor such as vi.
  2. Add the following line at the bottom of the file:

    export JAVA_OPTS="-Xms1024m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=356m -XX:PermSize=256m -XX:MaxPermSize=356m"

    We recommended that the Xms and Xmx values be identical so your memory block is contiguous.

MacOSX

When FileCatalyst Webmail or Workflow is installed on a MacOSX machine, the catalina startup script.

  1. Navigate to the /apache-tomcat-/bin/ folder using Finder.
  2. Edit the catalina.sh file in a text editor.
  3. Add the following line at the end of the file:

    export JAVA_OPTS="-Xms1024m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=356m -XX:PermSize=256m -XX:MaxPermSize=356m"

    We recommended that the Xms and Xmx values be identical so your memory block is contiguous.

Overview

Logs are one of the critical pieces of information requested by FileCatalyst Support. Below is a list of those viewed as most useful from a support perspective and their default locations. If support has requested logs for a given event please remember that they are rotated every 24 hours at the time set in the configuration file and archived with the dates shown in the name.


Environment

FileCatalyst Workflow v4.9 and later.

FileCatalyst Webmail v4.9 and later.


Resolution

Note: The following log locations are shown for FileCatalyt Workflow, if you would like to see them for FileCatalyst Webmail, replace the Workflow entries with name of the name of the Webmail folder.

FileCatalyst Workflow:

  1. Web Server Logs:

    • Windows Location<path to Tomcat>\logs\ 
    • Linux Location: /opt/tomcat/logs/ or <tomcat_home>/logs/
    • Mac OSX Location: <path to tomcat_home>/logs/

      • tomcat7-stderr.<yyyy-mm-dd>.log
      • tomcat7-stdout.<yyyy-mm-dd>.log
      • catalina.<yyyy-mm-dd>.log

        These logs will show us issues with the Tomcat Web Server and its service. Any errors or exceptions that are caught by the Tomcat engine will be stored in these files.

  2. FileCatalyst Workflow Application Logs:

    • Windows Location<path to Tomcat>\webapps\logs\workflow\
    • Linux Location: /opt/tomcat/webapps/logs/workflow or <tomcat_home>/webapps/logs/workflow/
    • Mac OSX Location: <path to tomcat_home>/webapps/logs/workflow/

      • webapp.log 

        This is a catch-all log, it will contain more granular information about the FileCatalyst Workflow deployment, file transfers and the background process that launch.

  3. Internal FTP Session Logs:

    • Windows Location:  <path to>\workflow-config\FTPSessionLogs\
    • Linux Location: <path to>/workflow-config/FTPSessionLogs
    • Mac OSX Location:  <path to>/workflow-config/FTPSessionLogs

      • The path to the Workflow-Config logs can be found on the About FileCatalyst page of the FileCatalyst Workflow deployment.
      • Format <user><JobID><YY:mm:dd>_<hh:mm:ss>.log

Overview

This article will provide an example of a PHP Script that parses out the information available in the Post URL call from the FileCatalyst Client Software/API. The URL call uses a POST command to send the list of files. It needs to be run from an Apache Tomcat Web Server with PHP Script on the same box as the Client Software/API. 

 

Environment

FileCatalyst Client Software/API v3.5 and later.

Apache Web Server with PHP v5.0 or later.

 

Resolution

This method can be used for various FileCatalyst Client Applications. The following is a HotFolder Example:

  1. Create a scheduled task in the FileCatalyst HotFolder.
  2. Go to the Post Task tab and under the Post URL field call the following URL, replacing the information between the <>:

    http://<HostIP_Address>:<Port>/<PHPScript_Name>.php

  3. The following sample script will write the contents to a file located in c:/temp/succeeded.txt. This path can be changed by modifying the $fileOutput parameter in the script.

    Sample PHP Script:

    <html>
        <?php
        /* Script to Output File Names Received By FileCatalyst HotFolder Author: John Tkaczewski */
       
        $filePostVariable = 'f'; //The POST variable containing the file listing no paths of only successful transfers
        $filePathPostVariable = 'lf'; //The POST variable containing the file listing with full paths of only successful transfers
        $fileSizesPostVariable = 's'; //The POST variable containing the file sizes 
        $fileStatusPostVariable = 'status'; //The POST variable containing the status for each file, see HF docs
        $fileAllFilesPostVariable = 'allfiles'; //The POST variable containing the file listing with full paths including files scheduled to be transferred and that failed to transfer
        $fileDelimiter = '|'; //The delimiter that seperates the file list, it's a pipe by default and can't be chnaged 
        $fileOutput = 'c:/temp/succeeded.txt'; //The file location to write the file list 
    
        
        //Initialize the output variable 
        $outputData = '';
    
        //Break the file list into an array 
        $filePost = $_POST[$filePostVariable];
        $filePost = explode($fileDelimiter, $filePost);
        $outputData .= "\n***** f parameter values****\n";
        if (is_array($filePost)) {
            //Loop through the file list 
            foreach ($filePost as $tmp) {
                //Reverse the string for tokenizing purposes 
                $tmp = strrev($tmp);
                //Tokenize the string 
                $fileName = strtok($tmp, "\\/");
                if ($fileName) {
                //If a filename exists, reverse it back to original state 
                    $fileName = strrev($fileName);
                    //Append the filename to the output variable 
                    $outputData .= $fileName . "\r\n";
                }
            }
        }
        $outputData .= "\n***** lf parameter values****\n";
        //Break the file list with paths into an array 
        $filePost = $_POST[$filePathPostVariable];
        $filePost = explode($fileDelimiter, $filePost);
       
        if (is_array($filePost)) {
            //Loop through the file list 
            foreach ($filePost as $tmp) {
                //Reverse the string for tokenizing purposes 
                $tmp = strrev($tmp);
                //Tokenize the string 
                $fileName = strtok($tmp, "\\/");
                if ($fileName) {
                //If a filename exists, reverse it back to original state 
                    $fileName = strrev($fileName);
                    //Append the filename to the output variable 
                    $outputData .= $fileName . "\r\n";
                }
            }
        }
        
        $outputData .= "\n***** s parameter values****\n";
        //Break the file sizes into an array 
        $filePost = $_POST[$fileSizesPostVariable];
        $filePost = explode($fileDelimiter, $filePost);
       
        if (is_array($filePost)) {
            //Loop through the file list 
            foreach ($filePost as $tmp) {
                //Reverse the string for tokenizing purposes 
                $tmp = strrev($tmp);
                //Tokenize the string 
                $fileName = strtok($tmp, "\\/");
                if ($fileName) {
                //If a filename exists, reverse it back to original state 
                    $fileName = strrev($fileName);
                    //Append the filename to the output variable 
                    $outputData .= $fileName . "\r\n";
                }
            }
        }
    
        $outputData .= "\n***** status parameter values****\n";
        //Break the file status into an array 
        $filePost = $_POST[$fileStatusPostVariable];
        $filePost = explode($fileDelimiter, $filePost);
        
        if (is_array($filePost)) {
            //Loop through the file list 
            foreach ($filePost as $tmp) {
                //Reverse the string for tokenizing purposes 
                $tmp = strrev($tmp);
                //Tokenize the string 
                $fileName = strtok($tmp, "\\/");
                if ($fileName) {
                //If a filename exists, reverse it back to original state 
                    $fileName = strrev($fileName);
                    //Append the filename to the output variable 
                    $outputData .= $fileName . "\r\n";
                }
            }
        }
        
        
        $outputData .= "\n***** allfiles parameter values****\n";
        //Break the file allfiles into an array 
        $filePost = $_POST[$fileAllFilesPostVariable];
        $filePost = explode($fileDelimiter, $filePost);
       
        if (is_array($filePost)) {
            //Loop through the file list 
            foreach ($filePost as $tmp) {
                //Reverse the string for tokenizing purposes 
                $tmp = strrev($tmp);
                //Tokenize the string 
                $fileName = strtok($tmp, "\\/");
                if ($fileName) {
                //If a filename exists, reverse it back to original state 
                    $fileName = strrev($fileName);
                    //Append the filename to the output variable 
                    $outputData .= $fileName . "\r\n";
                }
            }
        }
        
        //Create and/or open the file with write permissions 
        $handle = fopen($fileOutput, 'w');
    
        //Flush the output to the file 
        fputs($handle, trim($outputData));
    
        //Close the file 
        fclose($handle);
    
        /* End Script */
        ?> 
    
        <html> 
            <!--The HotFolder/Server ignores the response from the server. There is no need to redirect or output 
            anything besides the empty HTML tag--> 
        </html>

  4. Save the script above with the extension of PHP and use the saved filename in step 2.

Environment

FileCatalyst Direct Suite v3.0 and later.

 

Overview

Communication between the Client Side software and the FileCatalyst Server is always done through the Control Socket or Command Channel. This connection is usually configured over a non-secure Port such as 21 or a Port secured using SSL, such as 990. The login credentials and commands like, GET, PUT, LS and more.

Once the credentials are verified between the FileCatalyst Server and the Client Side software, a separate link is negotiated where the data flow takes place. Data does not normally flow back and forth over the Command Channel for FileCatalyst Products using UDP acceleration.

Exactly what range is configurable on the FileCatalyst Server?

By default, we specify the port range to be 8000-8999. You are free to change this as you see fit, however, keep in mind that for every command you send out (LS/DIR, PUT, GET, etc), one port needs to be opened up and closed. Thus, if you’re transferring 1000 small 1k files, you are spending all of your time opening and closing sockets (so it’s preferable to zip archive these files). It also means that if you limit the port range to a very small number (say, 10 ports), you are potentially setting yourself up for a port availability issue, even with 1 user connecting. Ports are released by Java relatively quickly, but the OS can hang onto some ports for a long time. These release times vary from OS to OS. The FileCatalyst server does recycle the port list, but it’s always a good idea to have more rather than less.



Overview

Some networks will have IPV6 enabled by default and this may cause an issue with licensing products. This is an issue that has been reported for customers using Unlimi-tech Applets and FileCatalyst Applets. 

Microsoft has posted a Knowledgebase Article on how to disable IPV6 globally and for a specific NIC. These solutions can be deployed automatically or manually.

The guide can be accessed here: https://support.microsoft.com/en-us/kb/929852


Environment

FileCatalyst Direct Suite v3.4 and later.

FileCatalyst Workflow v4.9 and later.

FileCatalyst Webmail v4.9 and later.

Unlimi-tech Applets all versions.

Windows Environment.

 

Resolution:

The following is an excerpt from the full solutions article using manual steps:

  1. Click Start, type regedit in the Start Search box, and then click regedit.exe in the Programs list.
  2. In the User Account Control dialogue box, click Continue.
  3. In Registry Editor, locate and then click the following registry subkey:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\

  4. Double-click DisabledComponents to change the entry.

    If the DisabledComponents entry is unavailable, you must create it. To do this, follow these steps:

    1. In the Edit menu, point to New, and then click DWORD (32-bit) Value.
    2. Type DisabledComponents, and then press Enter.
    3. Double-click DisabledComponents.

  5. Type the following values in the Value Data Field to configure the IPv6 protocol to the intended state, and then click OK:
    1. Type 0x10 to disable IPv6 on all non-tunnel interfaces (both LAN and Point-to-Point Protocol [PPP] interfaces).
  6. A restart of the machine will be required for the settings to be refreshed.

Overview

This article will provide a guide on how to enable the Java Console for MacOSX.

 

Environment

FileCatalyst Applets v3.4 and later.

FileCatalyst Workflow v4.9 and later.

FileCatalyst Webmail v4.9 and later.

Unlimi-tech Applets all versions.

MacOSX Environment.

 

Resolution

  1. After Java JDK or JRE are installed open System Preferences through Apple Menu.
  2. Open Java application.
  3. When the new window opens, navigate to the Advanced tab.
  4. Scroll down and select the Show Console radial.
  5. Click OK to save your changes.

Most information in this artcle came from the following link:
http://www.29west.com/docs/THPM/udp-buffer-sizing.html
Please give this page a read.

This article explains the impact of UDP buffer sizes on high speed transfers. It also tells how to increase the values for Linux.
Too little UDP buffer space causes the operating system kernel to discard UDP packets. The resulting packet loss has consequences described below.

Latency--The time that passes between the initial transmission of a UDP packet and the eventual successful reception of a retransmission is latency that could have been avoided were it not for the intervening loss.

Bandwidth--Assuming that the initial transmission was not frivolous, it's likely that UDP loss will result in retransmission. Bandwidth used for retransmissions may become significant, especially in cases where there is a large amount of loss or a large number of receivers experiencing loss.

CPU Time--UDP loss causes the receiver to use CPU time to detect the loss, request one or more retransmissions, and perform the repair. Note that efficiently dealing with loss among a group of receivers requires the use of many timers, often of short-duration. Scheduling and processing such timers generally requires CPU time in both the operating system kernel ("system time") and in the application receiving UDP ("user time"). Additional CPU time is required to switch between kernel and user modes.

On the sender, CPU time is used to process retransmission requests and to send retransmissions as appropriate. As on the receiver, many timers are required for efficient retransmission processing, thus requiring many switches between kernel and user modes.

Memory--UDP receivers that can only process data in the order that it was initially sent must allocate memory while waiting for retransmissions to arrive. UDP loss causes such receivers to receive data in an order different than that used by the sender. Memory is used to restore the order in which it was initially sent.

Perhaps the two most significant consequences of too much UDP buffer space are slower recovery from loss and physical memory usage.

Each of these is discussed in turn below.

Slower Recovery--To best understand the consequences of too much UDP buffer space, consider a stream of packets that regularly updates the current value of a rapidly-changing variable in every tenth packet. Why buffer more than ten packets? Doing so would only increase the number of stale packets that must be discarded at the application layer. Given a data stream like this, it's generally better to configure a ten-packet buffer in the kernel so that no more than ten stale packets have to be read by the application before a return to fresh ones from the stream.

It's often counter-intuitive, but excessive UDP buffering can actually increase the recovery time following a large packet loss event. UDP receive buffers should be sized to match the latency budget allocated for CPU scheduling latency with knowledge of expected data rates.

Physical Memory Usage--It is possible to exhaust available physical memory with UDP buffer space. Requesting a UDP receive buffer of 32 MB and then invoking ten receiver applications uses 320 MB of physical memory.

Assuming that an average rate is known for a UDP data stream, the amount of latency that would be added by a full UDP receive buffer can be computed as:

Max Latency = Buffer Size / Average Rate

Note: Take care to watch for different units in buffer size and average rate (e.g. kilobytes vs. megabits per second).

Assuming that an average rate is known for a UDP data stream, the buffer size needed to avoid loss a given worst case CPU scheduling latency can be computed as:

Buffer Size = Max Latency * Average Rate

Note: Since data rates are often measured in bits per second while buffers are often allocated in bytes, careful conversion may be necessary.

The kernel variable that limits the maximum size allowed for a UDP receive buffer has different names and default values by kernel given in the following :

Linux net.core.rmem_max 131071
Solaris udp_max_buf 262144
FreeBSD, Darwin kern.ipc.maxsockbuf 262144
AIX sb_max 1048576
Windows None we know of Seems to grant all reasonable requests

The examples in this table give the commands needed to set the kernel UDP buffer limit to 8 MB. Root privilege is required to execute these commands.

Linux sysctl -w net.core.rmem_max=8388608
Solaris ndd -set /dev/udp udp_max_buf 8388608
FreeBSD, Darwin sysctl -w kern.ipc.maxsockbuf=8388608
AIX no -o sb_max=8388608 (note: AIX only permits sizes of 1048576, 4194304 or 8388608)


Making Changes Survive Reboot
The AIX command given above will change the current value and automatically modify /etc/tunables/nextboot so that the change will survive rebooting. Other platforms require additional work described below to make changes survive a reboot.
For Linux and FreeBSD, simply add the sysctl variable setting given above to /etc/sysctl.conf leaving off the sysctl -w part.
We haven't found a convention for Solaris, but would love to hear about it if we've missed something. We've had success just adding the ndd command given above to the end of /etc/rc2.d/S20sysetup.


Interpreting the output of netstat is important in detecting UDP loss. Unfortunately, the output varies considerably from one flavor of Unix to another. Hence we can't give one set of instructions that will work with all flavors.

For each Unix flavor, we tested under normal conditions and then under conditions forcing UDP loss while keeping a close eye on the output of netstat -s before and after the tests. This revealed the statistics that appeared to have a relationship with UDP packet loss. Output from Solaris and FreeBSD netstat was the most intuitive; Linux and AIX much less so. Following sections give the command we used and highlight the important output for detecting UDP loss.

Detecting Solaris UDP Loss
Use netstat -s. Look for udpInOverflows. It will be in the IPv4 section, not in the UDP section as you might expect. For example:

IPv4:
udpInOverflows = 82427

Detecting Linux UDP Loss
Use netstat -su. Look for packet receive errors in the Udp section. For example:

Udp:
38799 packet receive errors

Detecting Windows UDP Loss
Use netstat -s. Look for Receive Errors in the UDP Statistics for IPv4 section. For example:

UDP Statistics for IPv4
Receive Errors = 131213

Detecting AIX UDP Loss
Use netstat -s. Look for fragments dropped (dup or out of space) in the ip section. For example:

ip:
77070 fragments dropped (dup or out of space)
7.9.5. Detecting FreeBSD and Darwin UDP Loss
Use netstat -s. Look for dropped due to full socket buffers in the udp section. For example:

udp:
6343 dropped due to full socket buffers



You can find configuration information for UDP buffers here:  https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html

Please give this page a read.


This article explains the impact of UDP buffer sizes on high-speed transfers. It also tells how to increase the values for Linux.
Too little UDP buffer space causes the operating system kernel to discard UDP packets. The resulting packet loss has consequences described below.

Latency--The time that passes between the initial transmission of a UDP packet and the eventual successful reception of a retransmission is latency that could have been avoided were it not for the intervening loss.

Bandwidth--Assuming that the initial transmission was not frivolous, it's likely that UDP loss will result in retransmission. Bandwidth used for retransmissions may become significant, especially in cases where there is a large amount of loss or a large number of receivers experiencing loss. 

CPU Time--UDP loss causes the receiver to use CPU time to detect the loss, request one or more retransmissions, and perform the repair. Note that efficiently dealing with loss among a group of receivers requires the use of many timers, often of short duration. Scheduling and processing such timers generally require CPU time in both the operating system kernel ("system time") and in the application receiving UDP ("user time"). Additional CPU time is required to switch between kernel and user modes.

On the sender, CPU time is used to process retransmission requests and to send retransmissions as appropriate. As on the receiver, many timers are required for efficient retransmission processing, thus requiring many switches between kernel and user modes.

Memory--UDP receivers that can only process data in the order that it was initially sent must allocate memory while waiting for retransmissions to arrive. UDP loss causes such receivers to receive data in an order different than that used by the sender. Memory is used to restore the order in which it was initially sent.

Perhaps the two most significant consequences of too much UDP buffer space are slower recovery from loss and physical memory usage. 

Each of these is discussed in turn below.

Slower Recovery--To best understand the consequences of too much UDP buffer space, consider a stream of packets that regularly updates the current value of a rapidly-changing variable in every tenth packet. Why buffer more than ten packets? Doing so would only increase the number of stale packets that must be discarded at the application layer. Given a data stream like this, it's generally better to configure a ten-packet buffer in the kernel so that no more than ten stale packets have to be read by the application before a return to fresh ones from the stream.

It's often counter-intuitive, but excessive UDP buffering can actually increase the recovery time following a large packet loss event. UDP receive buffers should be sized to match the latency budget allocated for CPU scheduling latency with knowledge of expected data rates. 

Physical Memory Usage--It is possible to exhaust available physical memory with UDP buffer space. Requesting a UDP receive buffer of 32 MB and then invoking ten receiver applications uses 320 MB of physical memory. 

Assuming that an average rate is known for a UDP data stream, the amount of latency that would be added by a full UDP receive buffer can be computed as:

    Max Latency = Buffer Size / Average Rate

Note: Take care to watch for different units in buffer size and average rate (e.g. kilobytes vs. megabits per second).

Assuming that an average rate is known for a UDP data stream, the buffer size needed to avoid loss a given worst case CPU scheduling latency can be computed as:

    Buffer Size = Max Latency * Average Rate

Note: Since data rates are often measured in bits per second while buffers are often allocated in bytes, careful conversion may be necessary.


The kernel variable that limits the maximum size allowed for a UDP receive buffer has different names and default values by kernel given in the following :

Default UDP buffers:
Linux net.core.rmem_max 131071 
Solaris udp_max_buf 262144 
FreeBSD, Darwin kern.ipc.maxsockbuf 262144 
AIX sb_max 1048576 


Windows None we know of Seems to grant all reasonable requests 

The examples in this table give the commands needed to set the kernel UDP buffer limit to 8 MB. Root privilege is required to execute these commands.

Recommended UDP Buffers (8+MB):
Linux sysctl -w net.core.rmem_max=8388608 
Solaris ndd -set /dev/udp udp_max_buf 8388608 
FreeBSD, Darwin sysctl -w kern.ipc.maxsockbuf=8388608 
AIX no -o sb_max=8388608 (note: AIX only permits sizes of 1048576, 4194304 or 8388608) 


Making Changes Survive Reboot
The AIX command given above will change the current value and automatically modify /etc/tunables/nextboot so that the change will survive rebooting. Other platforms require additional work described below to make changes survive a reboot.

For Linux and FreeBSD, simply add the sysctl variable setting given above to /etc/sysctl.conf leaving off the sysctl -w part.

We haven't found a convention for Solaris, but would love to hear about it if we've missed something. We've had success just adding the ndd command given above to the end of /etc/rc2.d/S20sysetup.


Interpreting the output of netstat is important in detecting UDP loss. Unfortunately, the output varies considerably from one flavor of Unix to another. Hence, we can't give one set of instructions that will work with all flavors.

For each Unix flavor, we tested under normal conditions and then under conditions forcing UDP loss while keeping a close eye on the output of netstat -s before and after the tests. This revealed the statistics that appeared to have a relationship with UDP packet loss. Output from Solaris and FreeBSD netstat was the most intuitive; Linux and AIX much less so. Following sections give the command we used and highlight the important output for detecting UDP loss.

Detecting Solaris UDP Loss
Use netstat -s. Look for udpInOverflows. It will be in the IPv4 section, not in the UDP section as you might expect. For example:

IPv4:
udpInOverflows = 82427

Detecting Linux UDP Loss
Use netstat -su. Look for packet receive errors in the Udp section. For example:

Udp:
38799 packet receive errors

Detecting Windows UDP Loss
Use netstat -s. Look for Receive Errors in the UDP Statistics for IPv4 section. For example:

UDP Statistics for IPv4
Receive Errors = 131213

Detecting AIX UDP Loss
Use netstat -s. Look for fragments dropped (dup or out of space) in the ip section. For example:

ip:
77070 fragments dropped (dup or out of space)
7.9.5. Detecting FreeBSD and Darwin UDP Loss
Use netstat -s. Look for dropped due to full socket buffers in the udp section. For example:

udp:
6343 dropped due to full socket buffers

Overview

Every deployment of FileCatalyst Workflow or Webmail is shipped with an Internal FTP only FileCatalyst Server. The default path where the storage points to is within the Workflow or Webmail configuration folder. It is important that the FTP data and deployment configurations should be separated, for various reasons. 

The resolution method in this article can only be performed in Firefox or Internet Explorer as the usage of a Java Applet is required.



Environment

FileCatalyst Webmail v4.9 and later.

FileCatalyst Workflow v4.9 and later.

Firefox and Internet Explorer Web Browsers.

 

Resolution

  1. Log into FileCatalyst Workflow or Webmail as the Super Admin User. Be default, the account name is init.
  2. Navigate to the Modify Configuration button on the top menu bar.
  3. Click on the link for FC Servers / Script Execution, under the System Configuration section.
  4. Expand the Server details for the Internal Server and click Launch Admin Applet link. If launching Java applet is not possible, you can also download the Remote Admin application from our download site and connect to the Internal FC Server using the Remote Admin app.
  5. In another window, the FileCatalyst Remote Admin applet will load. Accept any Java security prompts that are displayed.
  6. When the applet loads, locate the doctera account and hit Edit.
  7. Under the Account Settings tab, enter the path to your Home Directory.
  8. Hit OK to apply the changes.
  9. Go back to the window that is logged in as the Super Admin User and click on Modify Configuration.
  10. Under the System Restart section, click the Restart System link.

Overview

Azure is Microsoft’s Cloud Platform that enables applications to be deployed across a high availability Internet-hosted environment. To deploy a FileCatalyst application, the proper ports must be opened and accessible to your Azure environment. This article will discuss the ports that need to be opened and how to open these ports using a scripted method.

As of the writing of this article, Azure only allows a total of 150 ports to be opened for any deployment. This is extremely important to note.

The default ports for FileCatalyst Direct are:

  • Port 21 for TCP.
    This is used as the communication channel. Port 21 is a default specified in the FileCatalyst Server and is not secure. You can change this as long as both FileCatalyst Server and Client Applications are using the same port.

  • Port 22 for TCP.
    This port is used for SSH connections.

  • Port 990 for TCP.
    This Port is secured over SSL and is also used as the communication channel. The default value of 990 can also be changed in the FileCatalyst Server Remote Admin Application.

  • Port range 8000-8999 for TCP and UDP.
    Some firewalls and NAT devices require setting up 2 separate rules and others allow you to specify TCP and UDP. These ports are used to transfer the Data to and from the FileCatalyst Server and is commonly referred to as the Data Channel. 
  • Port 12400 for TCP.
    This is the Remote Admin Port. This port is exclusive to the FileCatalyst Server Remote Admin Application.

The default ports for FileCatalyst Workflow/Webmail are:

  • Port 80 for TCP.
    HTTP inbound connections are accepted by the Tomcat Web Server on this port. 
  • Port 443 for TCP.
    HTTPS connections and secured communication to the Tomcat Web Server use this port. 

 

Environment

Microsoft Azure Cloud Computing

FileCatalyst Direct Suite v3.5 and later.

FileCatalyst Workflow v4.9.4 and later.

FileCatalyst Webmail v4.9.4 and later.

Windows and Linux Environments.

 

Resolution

  1. Plan of ports to open.
    Your local Windows System will need to allow data connection ports. Windows Firewall is accessible from Control Panel. Microsoft Azure has a limit of 150 open ports. For FileCatalyst, we will configure 70 ports for file transfers. Data Ports will need to be opened for both TCP and UDP.

  2. Create a folder on your local computer called c:\AzureScripts.

  3. PowerShell Prerequisites:
    1. Change your working directory to the AzureScripts location.
    2. Download and install the Windows PowerShell and Azure command-line interface tools from Azure PowerShell Web Installer. (http://go.microsoft.com/fwlink/p/?linkid=320376&clcid=0x409)
    3. Open Windows Azure PowerShell application. You may have a Windows PowerShell installed. Do not use PowerShell that is shipped with Windows as it does not have the Azure libraries. Run the following commands:

      Add-AzureAccount

      This will open a dialog for you to sign in to the Azure Portal.

      Get-AzurePublishSettingsFile

      When your browser opens, a credentials.publishsettings file will download to your default download directory.

  4. Opening ports on Microsoft Azure using PowerShell with Azure Libraries.
    At first setup, you will not be able to execute scripts in PowerShell scripts. In order to run PowerShell scripts, run the following command:

    Set-ExecutionPolicy RemoteSigned

  5. Open the ports using PowerShell:
    1. Create a folder on your local computer called C:\AzureScripts or change your working directory to it if the folder exists.
    2. Copy the publish profile settings file you downloaded in prerequisite 3c into this folder.
    3. Download and copy the AddFCPortsToAzureFirewall.ps1 and FCPorts.csv files from the supporting assets archive included in this document to the AzureScripts folder.
    4. Edit the AddFCPortsToAzureFirewall.ps1 using a text editor and enter the appropriate information under Get-AzureVM -ServiceName [YOUR_INSTANCE_NAME].
    5. Save the file once you have made your changes.
    6. Open Windows Azure PowerShell application. 
    7. Change the working directory to the AzureScripts folder.
    8. Run the AddFCPortsToAzureFirewall.ps1 file.
    9. This should add all the ports needed by FC to your virtual instance.

 

Overview

Every instance that is deployed on Amazon EC2 must have Firewall and specific ports configured. By default, most of the ports that are needed are not open for TCP or UDP connections. The FileCatalyst Server, FileCatalyst Workflow, and FileCatalyst Webmail deployments all need to have their respective ports opened for connectivity and data transfer.

This article will walk you through a typical security group setup for an Amazon EC2 instance.

The default ports for FileCatalyst Direct are:

  • Port 21 for TCP.
    This is used as the communication channel. Port 21 is a default specified in the FileCatalyst Server and is not secure. You can change this as long as both FileCatalyst Server and Client Applications are using the same port.

  • Port 22 for TCP.
    This port is used for SSH connections.

  • Port 990 for TCP.
    This Port is secured over SSL and is also used as the communication channel. The default value of 990 can also be changed in the FileCatalyst Server Remote Admin Application.

  • Port range 8000-8999 for TCP and UDP.
    Some firewalls and NAT devices require setting up 2 separate rules and others allow you to specify TCP and UDP. These ports are used to transfer the Data to and from the FileCatalyst Server and is commonly referred to as the Data Channel. 
  • Port 12400 for TCP.
    This is the Remote Admin Port. This port is exclusive to the FileCatalyst Server Remote Admin Application.

  • Port 12480 for TCP.
    The internal Web Server uses this port to broadcast all communications. The Admin Applet, Link, and Servlet are hosted from this Web Server.

The default ports for FileCatalyst Workflow/Webmail are:

  • Port 80 for TCP.
    HTTP inbound connections are accepted by the Tomcat Web Server on this port. 
  • Port 443 for TCP.
    HTTPS connections and secured communication to the Tomcat Web Server use this port. 

 

Environment

FileCatalyst Direct Suite v3.5 and later.

FileCatalyst Workflow v4.9.5 and later.

FileCatalyst Webmail v4.9.4 and later.

Linux and Windows Environments.

 

Resolution

 

  1. Log into your AWS Console. Click on Security Groups on the left-hand side.



  2. Click on the Create Security Group button, see the attached illustration below.



  3. Specify the Security Group Name and Description.



  4. Hit Add Rule to add your first port. Here is an example of adding a single port 21 on TCP for a connection originating from anywhere.



  5. This is an example of how to add a range of ports. We have added the FileCatalyst Data Port range for both TCP and UDP.




  6. Once all your Firewall Rules have been added, hit the Create button.

  7. Using the Security Group Name, you can add this of Firewall Rules to your EC2 Instance.
Note:

Quick Start Guide: How to add External File System Storage on the FileCatalyst Server

Environment

Ubuntu 14.04 on AWS EC2.

FileCatalyst Direct Server v3.7.2 Build 6 and newer.


Deployment Guide

From the AWS Marketplace search for FileCatalyst and from the results select “FileCatalyst Direct Server Hourly Billing”.

  1. To create and launch an instance click on Continue to Subscribe. You will need to have an AWS Account to continue.

    Use the 1-Click Launch option to begin your deployment. It is fairly easy to setup and most of our recommended settings are displayed here for you to select and modify:



  2. If there is more than one version of the FileCatalyst Direct Server available you will be able to access it from the Version menu. You can view the Release Date of the software on the right of your selection:


    Please note that this will spin up FileCatalyst in an Ubuntu OS. We are going to use v3.7.2 build 6 for this document.

  3. When using the 1-Click Launch option, AWS selects a Region for you, however, if you would like to change the location of your EC2 instance you can modify the selection by using the drop-down list:



  4. Our software has a specific set of requirements for installation and deployment. We have provided a list of compatible instances that you can use to deploy your FileCatalyst Server which is compatible with this AMI. The EC2 instance specifications will be displayed once you make a selection:


    The instance size determines how much bandwidth the OS is allocated by AWS and in turn, the license limit applied to the FileCatalyst Server. Here is a table that shows you the FileCatalyst License bandwidth limit

    Instance Size Bandwidth License
    Large 500Mbps
    XLarge 750Mbps
    2XLarge 1Gbps
    4XLarge 2Gbps
    10XLarge 5Gbps
    16XLarge 10Gbps

    Note: The maximum speed achieved depends on a number of factors, disk speed, networking options etc. Contact support for additional assistance when trying to achieve high speeds.
  5. Networking options are preset in this installation. You can either use the preset values under the VPC Settings tab or create your own VPC and Subnet(s). Please make sure your VPC has connectivity from the outside.



    Please refer to AWS VPC Guide for more information. These settings can be modified after you launch the instance.

  6. Inbound and Outbound Firewall Rules will need to be added so that you can have traffic and data flow in and out of your deployment. We have already created some firewall rules for you based on our default configurations. We recommend that you start here, this will allow you to get up and running fast and as with any settings in the AWS platform they can be changed at a later date. Please select “Create new based on seller settings”:



  7. Create a key pair through your EC2 Console to access the Ubuntu OS through PuTTY. Please click on the link to proceed on the deployment page:


    After your browser is refreshed you will be able to view your newly created Key Pair:


    Check your browsers download location for the PEM file that was downloaded.

  8. To launch your instance you will need to accept the charges, terms and the EULA:


  9. From our EC2 Console connect to the instance using PuTTY (If on Windows) or using any SSH client. Right click on your instance and hit connect. 

    Use this guide if on Windows on Connecting to Your Linux Instance from Windows Using PuTTY

  10. On your first connection, the FileCatalyst Server will already be installed, licensed and running. You will need to enable the Masquerade feature in the FileCatalyst Server so that external clients can connect to the deployment using the Public IP.
    In the PuTTY terminal look inside /home/ubuntu/ for a README.txt. This file will contain instructions and credentials which you can use to connect to the FileCatalyst Server.

    You can access the Download Portal or choose FileCatalyst Server Admin for your OS:

    MacOSX v3.7.2 build 6 package installer
    Windows v3.7.2 build 6 executable installer 
    Linux v3.7.2 build 6 zip package
    Contact [email protected] for download credentials. Please make sure the build is v3.7.2 Build 6 when downloading the Server Admin.



  11. Open your FileCatalyst Server Remote Admin and click on the Advanced tab. In the first row, you will see the Network tab, click on it.
    Locate the IP Settings section and Enable Masquerade Address. Enter your Public IPv4 address in the Masquerade Address field and hit Apply.



    This step will allow the FileCatalyst Server to broadcast the Public IP of the machine rather than the Internal IP.
Note:

Quick Start Guide: How to add External File System Storage on  the FileCatalyst Server

Overview

This guide will provide a quick walkthrough on launching your own FileCatalyst Server Per Hour Billing instance.

Environment

Ubuntu 16.04
FileCatalyst Server v3.7.2

Deployment Guide

Once you have setup the FileCatalyst Direct Server Per Hour Billing from the Azure Marketplace you can use the following steps to get your deployment online:

  1. Using the link above in the Azure Marketplace, click on Get It Now to begin your subscription. You will be required login to your Azure account.



  2. Once you are logged in you can commence the deployment process by clicking create:



  3. In the basic configuration tab, you will need to specify connection details and a connecting username. For this article, we have used ubuntu as the username and specified a password. You can choose the options that fit your needs. Hit Ok to continue to the next tab.



  4. In the next step, you will configure the type of instance you would like to use. You will need to choose the best VM option depending on the amount of data (number of files and total size), number of simultaneous connections, compression settings and other applications that are running on the machine. You can reference our System Requirements and configure an instance you need.

  5. Once you have configured any additional options and accepted the terms and agreements your instance will be created on the Azure Dashboard. Click Connect on your instance to get SSH connection details. 



  6. Connect a Terminal or PuTTY session to your Azure Deployment and sudo to the root user:

    sudo -i
  7. Change your working directory so that you are in the FileCatalyst Server installation path using:

     cd /opt/utechsoft/server 

  8. Set execute permissions on the launch scripts. Make sure your current working directory is still /opt/utechsoft/server/. Run the following command to set the permissions:

    chmod u+x *.sh
  9. Once you are in, you will need to stop the FileCatalyst Server service if it is running. 

    service fcserver stop

  10. (Optional) The default admin password is: movefilesfast

    To change the Remote Administration password run the following command:

    java -jar FileCatalystServer.jar -passwdadmin

    The output should look like this:



  11. Start the FileCatalyst Server service using this command:

    service fcserver start

  12. The FileCatalyst Server is now running in headless mode. You can administer the Server remotely using the FileCatalyst Server Remote Admin tool. 
    You can access the Download Portal or choose FileCatalyst Server Admin for your OS:

    MacOSX v3.7.2 package installer
    Windows v3.7.2 executable installer 
    Linux v3.7.2 zip package

    Contact [email protected] for download credentials. 


  13. Open your FileCatalyst Server Remote Admin and click on the Advanced tab. In the first row, you will see the Network tab, click on it. 
    Locate the IP Settings section and Enable Masquerade Address. Enter your Public IPv4 address in the Masquerade Address field and hit Apply.



    This step will allow the FileCatalyst Server to broadcast the Public IP of the machine rather than the Internal IP. You can obtain the Public IP of your deployment from the Azure Dashboard.
Note:

Quick Start Guide: How to add External File System Storage on  the FileCatalyst Server

Overview

When a connection comes into the Tomcat Web Server it can be directed to use an HTTPS Socket rather than a non-secure HTTP Socket. Tomcat allows a configuration where HTTPS can be enforced only by the Workflow or Webmail deployment or a server wide rule can be enforced.

This article will discuss the how to redirect connections to HTTPS using the Tomcat Web Server. It is not necessary to use a server wide rule and an application wide rule. 


Environment

FileCatalyst Workflow v4.9 and later.

FileCatalyst Webmail v4.9 and later.

Tomcat Web Server v7.0 and later.

 

Resolution

Server Wide Rule:

  1. Shutdown the Tomcat Web Server Service.
  2. Locate the web.xml file which is located in /<Tomcat Root>/conf/web.xml folder.
  3. Edit the file web.xml and add the following to the end of the file just before the </web-app> tag:

    <!-- redirect all traffic to the SSL port -->
    <security-constraint>
      <web-resource-collection>
        <web-resource-name>Automatic SLL Forwarding</web-resource-name>
        <url-pattern>/*</url-pattern>
      </web-resource-collection>
      <user-data-constraint>
        <transport-guarantee>CONFIDENTIAL</transport-guarantee>
      </user-data-constraint>
    </security-constraint>

  4. Save the file and restart the Tomcat Web Server for the changes to take effect.

Application Wide Rule:

  1. Shutdown the Tomcat Web Server Service.
  2. Locate the web.xml file for the Webmail or Workflow deployment which is located in /<Tomcat Root>/webapps/workflow/WEB-INF/web.xml or /<Tomcat Root>/webapps/fcweb/WEB-INF/web.xml.
  3. Edit the file web.xml and add the following to the end of the file just before the </web-app> tag:

    <security-constraint>
       <web-resource-collection>
        <web-resource-name>Protected Context</web-resource-name>
          <url-pattern>/*</url-pattern>
      </web-resource-collection>
      <!-- auth-constraint goes here if you requre authentication -->
      <user-data-constraint>
        <transport-guarantee>CONFIDENTIAL</transport-guarantee>
      </user-data-constraint>
    </security-constraint>

  4. Save the file and restart the Tomcat Web Server for the changes to take effect.

Set Redirect Port for Tomcat:

Any HTTP connections to your FileCatalyst Webmail or Workflow deployment will be redirected to HTTPS instead using the Connector redirect port set in server.xml.
Default installations of Tomcat have this redirectPort set to 8443. If this is not the HTTPS port you are using, you will need to update server.xml accordingly. 

Note:

If you modify the application specific web.xml, you will have to remember to re-insert this code every time you upgrade, because the upgrade procedure requires deleting the entire webapp folder including the WEB-INF/web.xml file.

 

Overview

The FileCatalyst Suite of products can leverage cloud resources supplied by Amazon AWS. Using EC2 Instances, users are able to create and deploy our software on many different versions of Windows and Linux Operating Systems. You will need to consult Amazon AWS for their Cloud Services Pricing.

The products in the FileCatalyst Direct Suite can be downloaded directly from our Download Page. To access the page, you will need credentials which can be obtained from your FileCatalyst Sales Representative or the FileCatalyst Support Team. The FileCatalyst Direct Download Page can be accessed through this link: http://filecatalyst.software/fc-direct-download.html

The following guide will walk you through the basic setup of an AWS EC2 Instance, the configuration of security and network groups and the deployment of FileCatalyst Direct.

Environment

FileCatalyst Direct Suite v3.6 and later.

Amazon AWS EC2 Instances.

Resolution

Use the following steps to ensure that you have full connectivity in and out of your EC2 Instance. The following steps may redirect you to other guides, please complete them in the order they appear in this section.  

Some of the links below will require credentials to access the respective material. If you do not have these credentials please contact your FileCatalyst Sales Representative or the FileCatalyst Support Team.

  1. Choosing the right EC2 Instance and Operating System 

    Based on the System Requirements (http://filecatalyst.com/resources/system-requirements/), you can choose the EC2 Instance that fits the needs of your deployment.  When prompted to select the architecture type please select "64-bit (x86)". Using the "64-bit (Arm)" option will not launch the Server properly.
  2. Security Groups and Network Configuration

    You can use the following guide on FileCatalyst Amazon EC2 Port Configuration and Security Groups Guide: http://support.filecatalyst.com/index.php?/Knowledgebase/Article/View/263/0/firewall-and-port-configuration-for-amazon-ec2-instances-using-security-groups

    It is very important that the network is properly configured so that you do not encounter any issues that are firewall related when transferring data in or out of Amazon. 

  3. Installation Instructions 

    Once you have downloaded the respective application you would like to deploy to your EC2 Instance, you can use one of the following Quickstart Guides to install your FileCatalyst Application:

Notes:


There are other common articles that will be useful in customizing your deployment. You can access them using the links below:

In the External File Systems document, there is a list of limitations when using Cloud Storage (S3 buckets) which are imposed by Amazon. You can view them here: http://filecatalyst.software/download/filesFCDirect/current/documentation/server/efs.html#implementationNotes

 

Overview

Before moving your FileCatalyst Workflow deployment from staging to live status, there are a few optional steps that can be taken to ensure that your Tomcat Web Server has been hardened. The Hardening of a Web Server is a process where the security on the Web Server is enhanced by applying specific changes to the configuration. This results in a reduced risk of malicious attacks on the Web Server.

Many of the steps shown below have been extracted from https://wiki.owasp.org/index.php/Securing_tomcat. Please visit this site to get more information.

This article will guide you through some of the sections of the Tomcat Server. The paths and configurations mentioned below are referenced based on Windows OS, however, the same configurations can be used on a Linux OS. 

 

Environment

FileCatalyst Workflow v5.0 and newer.

Apache Tomcat Web Server

Note:
These modifications should be applied to an external installation of Apache Tomcat and not to any instances that have used the self-executable installer. 

 

Resolution

In order to successfully harden Apache Tomcat, you will need to resolve all of the issues below.

  1. Removing Server Banner or Server Name

    The Server Name is usually returned in the Response Headers. The Response Header can be customized to not return the name or type of Web Server that is being used.

    If you hit the landing page of FileCatalyst Workflow (http://www.yoursite.com/workflow) and Right Click on the page and select Inspect Element. You will find a docked window inside your browser, navigate to the Network Tab and reload the page. Click on the logon.jsp link and look for the Response Headers section.  A typical (default) response looks like:

    Content-Type:text/html;charset=UTF-8
    Date:Fri, 08 Jul 2016 14:36:40 GMT
    Server:Apache-Coyote/1.1
    Transfer-Encoding:chunked
    X-FRAME-OPTIONS:SAMEORIGIN

    Use the following steps to remove Server:Apache-Coyote/1.1 entry:

    1. Shutdown the Tomcat Web Server.
    2. Using a text editor open the Server.xml file. It is typically found in <Tomcat_Home>/conf/server.xml.
    3. Scroll down to your Connector Port add the following:

      Server=" " 

      Your modified Connector will look like:

      <Connector port="80" protocol="HTTP/1.1"
      connectionTimeout="20000"
      Server=" "
      redirectPort="8443" />

    4. Save the file and close the text editor.
    5. Start your Tomcat Web Server and the new changes will take effect.
    6. The new Response Header that is returned will have the following information:

      Content-Type:text/html;charset=UTF-8
      Date:Fri, 08 Jul 2016 14:45:55 GMT
      Server:
      Transfer-Encoding:chunked
      X-FRAME-OPTIONS:SAMEORIGIN
  2. Use SSL connections and force all connections to use HTTPS.

    1. For the Tomcat Web Server to accept SSL connections, a valid SSL Certificate needs to be deployed on the Web Server. Use the following guide to Generate and deploy the SSL Certificate to the Tomcat Server:
      http://support.filecatalyst.com/index.php?/Knowledgebase/Article/View/295/9/tomcat-csr-and-ssl-certificate-installation-and-renewal

    2. Once the SSL Certificate is deployed, all connectivity to the Web Server can be forced to use HTTPS. The steps to modify the configurations on the Web Server can be found here:
      http://support.filecatalyst.com/index.php?/Knowledgebase/Article/View/317/0/how-to-force-https-connections-in-workflow-or-webmail

  3. Run Tomcat from a non-privileged Account.

    It is best practice to use a separate account that has lowered permissions. This will protect other services running on the machine in case of any security breach.

    1. Create a user for the Tomcat Web Server.
    2. Linux - Change the ownership of the /<Tomcat_Home>/ directory to the newly created user.
      Windows - From the Service Manager, change the Tomcat Service properties to use another Logon Account.

  4. Removing unwanted applications the Webapps Directory.

    By default, Tomcat installs and deploys web applications which may not be required in your environment. These can be removed from Webapps directory (/<Tomcat_Home>/webapps/).

    1. The following folders may or may not be present but are not required for a FileCatalyst Workflow deployment:

      • Docs - This folder contains the Tomcat Documentation.
      • Examples - JSP and Servlets for demos are in this folder.
      • Manager - Web UI that allows deploy and deploy applications.

  5. Shutdown Port and Command

    The Tomcat Web Server is configured to be shut down on port 8005 and the command that is used is SHUTDOWN. Telnet to IP:port can be used to call a SHUTDOWN to the Tomcat Web Server.
    The port value and command string should be modified from the default values in the Server.xml file.

    1. Shutdown the Tomcat Web Server.
    2. Using a text editor open the Server.xml file and modify the following entry:

      <Server port="8005" shutdown="SHUTDOWN">

      These values should be changed to a Port value that is not in use and the command can be customized as well. Here is an example:


      <Server port="9889" shutdown="SHUTD0WNT0MCAT">

    3. Save the file and close the text editor.

  6. Change the Website Icon

    By default, the favicon.ico file located in the <Tomcat_Home>/webapps/ROOT/ folder will be loaded when anyone loads the FileCatalyst Workflow page or any other Tomcat Web Server page that is hosted on your machine. Replace this file (<Tomcat_Home>/webapps/ROOT/favicon.ico) with an icon of your choice and rename your file to favicon.ico and restart the Tomcat Web Server.

  7. Hide details from error pages
    Edit <tomcat_home>/conf/server.xml add the following valve to the host portion of the file:

    <valve classname="org.apache.catalina.valves.ErrorReportValve" showreport="false" showserverinfo="false"/>

Overview

This guide will provide a quick walkthrough on launching your own FileCatalyst Server and licensing it on an Amazon EC2 instance.

Environment

Amazon Linux Amazon Linux - 64-bit Amazon Machine Image
FileCatalyst Server v3.8

Deployment Guide

Launching FileCatalyst Server from the AWS Marketplace.

  • Select the version of the FileCatalyst Server you would like to use. We recommend using the option with the latest Release Date:

  • Choose the Region that you would like to use from the drop-down list.



  • Skip ahead one section and select the VPC Settings you would like to use.

  • When initially deploying the FileCatalyst Server, we strongly recommend that you use the preconfigured Security Group called "Create new based on seller settings". You will be able to modify these settings on your EC2 dashboard later.

  • To connect to the machine you will need a KeyPair. If there is an existing KeyPair you would like to use select it from the drop-down list.

  • Review the rest of your settings and Machine connection information on the "Launch this software" page and click on Launch at the bottom. 



Connecting to your FileCatalyst Server

  • From your EC2 dashboard, you will be able to see the instances that you have currently active. When the instance has completed set up (Status checks 2/2), select it and click on the "Connect" button at the top. This will show you a pop up on how to connect to your instance using PuTTY.


Amazon has provided documentation to assist with your connection to the Ubuntu deployment here: https://docs.aws.amazon.com/console/ec2/instances/connect/docs

Getting a license key

  • Connect to the FileCatalyst Server using PuTTY or a Terminal. Switch to the elevated root user using:

    sudo su

  • Change your working directory so that you are in the FileCatalyst Server installation path by executing:

    cd /opt/utechsoft/server

  • Once you are in the folder, you will need to stop the FileCatalyst Server service if it is running.

    service fcserver stop

    Your expected output should be:



  • Set execute permissions on the launch scripts in the installation folder. Make sure your current working directory is still /opt/utechsoft/server/. Run the following command to set the permissions:

    chmod u+x *.sh

  • The next step is to install a license key on the FileCatalyst Server. Run ./fc_server_console.sh and the FileCatalyst Server will attempt to start up. At this point, a request string will be shown:



    Send the request string to your Sales Rep at FileCatalyst indicating what type of license you would like and how many concurrent connections you need. They will send you a key with the appropriate features enabled. Please copy and paste the Request String. Taking a screen capture of the request string may delay you getting a License key. 

    If you do not have the contact for a Sales Rep please send the Request String to [email protected]

Applying the license key

  • Make sure your FileCatalyst Server is off before you proceed. You can run ps -ef | grep java in the terminal and make sure no instance is running. If you do see that there is an instance running please stop the application.

  • Under /opt/utechsoft/server directory, edit fcconf.conf in a text editor. Paste your license key you received after the = sign:

    ## License key

    FCServer.server.config.license=<Enter Key Here>

  • Save the file.

Recommended: Change the Remote Admin Password

  • To change the Remote Administration password run the following command from /opt/utechsoft/server:

    java -jar FileCatalystServer.jar -passwdadmin

    The output should look like:


Launching the FileCatalyst Server

 

Enable Masquerade Address


The FileCatalyst Server itself now needs to know that it's working through a NAT device. To enable this:

  • Open the Server Remote Admin Application.
  • Select Advanced on the left-hand side.
  • Check the box to Enable Masquerade Address.
  • In the address field, enter the Public IP. This is not the IP of the machine running the FileCatalyst Server. If the network device acts as a gateway between your network and the public internet, you can find the public internet address quite easily by browsing from any machine to this site: http://whatismyip.com 
    Alternatively, you can obtain your Public IP (IPv4) from your EC2 instance management console. Do not use your DNS.
  • Hit Apply.



Note:

Quick Start Guide: How to add External File System Storage on the FileCatalyst Server

Overview

After the installation of both client and server apps are complete, there may be transfer issues and/or connection issues. This may be a firewall and port issue.

If the following scenario sounds familiar, this article may be for you:

Scenario:


You have run through the installers on both the client and server side and have done what seems to be the required work (creating user accounts, added storage devices, etc) to initiate a file transfer. The file transfer fails. At this point you may have recognized that port 21 (assuming that the defaults are used) needs to get through your firewall and/or NAT, have made the administrative changes and tried again. It looks like a connection is now made, but the file transfer itself fails.

 

Both stages are symptomatic of firewall/NAT issues, and unfortunately, there's no way we can make things easier from the FileCatalyst application itself.  Before FileCatalyst can transfer files, some work needs to be done by the network administrator with regard to the ports, which need to be able to get through the firewall and forward properly in the NAT.

 

Environment

FileCatalyst Server v3.4 and later.

FileCatalyst HotFolder v3.4 and later.

FileCatalyst Express v3.4 and later.

 

Resolution

The following ports should be opened:

  • Port 21 for TCP.
    This is used as the communication channel. Port 21 is a default specified in the FileCatalyst Server and is not secure. You can change this as long as both FileCatalyst Server and Client Applications are using the same port.

  • Port 990 for TCP.
    This Port is secured over SSL and is also used as the communication channel. The default value of 990 can also be changed in the FileCatalyst Server Remote Admin Application.

  • Port range 8000-8999 for TCP and UDP.
    Some firewalls and NAT devices require setting up 2 separate rules and others allow you to specify TCP and UDP. These ports are used to transfer the Data to and from the FileCatalyst Server and is commonly referred to as the Data Channel.
     
  • Port 12400 for TCP.
    This is the Remote Admin Port. This port is exclusive to the FileCatalyst Server Remote Admin Application.

  • Port 12480 for TCP.
    The internal Web Server uses this port to broadcast all communications. The Admin Applet, Link and Servlet are hosted from this Web Server.
  • Enable Masquerade Address (if behind NAT device)
    The FileCatalyst Server itself now needs to know that it's working through a NAT device. To enable this:
    1. Open the Server Remote Admin Application.
    2. Select Advanced on the left-hand side.
    3. Check the box to Enable Masquerade Address.
    4. In the address field, enter the Public IP. This is not the IP of the machine running the FileCatalyst Server. If the network device acts as a gateway between your network and the public internet, you can find the public internet address quite easily by browsing from any machine to this site: http://whatismyip.com 
    5. Hit Apply.

 

 

Notes

  1. With regards to NAT, if you are behind any sort of device (ie a router) that forwards ports, these need to be configured as well. The exact same ports and protocols as the firewall need to be forwarded from the device to the machine that hosts the FileCatalyst Server.

  2. A few other troubleshooting notes:

    1. Check your processes and make sure you are not accidentally running multiple instances of FileCatalyst Server or the Server Remote Admin Application, which will conflict with each other. This also applies to the FileCatalyst Hotfolder and HotFolder Remote Admin Application.

    2. Make sure that another application is not trying to use the same ports. Most commonly, a pre-existing FTP server will already be bound to Port 21 and sometimes Port 22.

    3. Verify that an unexpected extra firewall or intrusion detection device is not in the way. Some organizations' IT departments have created security scenarios in which Windows (or other built-in) Firewalls are meant to be disabled. If these firewalls are inadvertently re-enabled, the Command Channel or Data Channel might get blocked.

Overview

All of our FileCatalyst Products are installed with default memory limits, including both initial reserve and maximum memory. However, these can be configured to fit your specific needs and environment.  This article will go through the configuration process for each FileCatalyst Product for Windows, Linux, and Mac OSX

NOTE:  

  • For all environments:
    If you are using a 32-bit Operating System or Java 32-bit, the maximum memory you can define for Java is limited to 1.5GB of memory.  64-bit Operating Systems running 64-bit Java can allocate a maximum of 4 GB.
  • It is also recommended (but not required) to set the minimum and the maximum memory to the same value, as this allows JAVA to allocate a single continuous block of the memory from the OS on startup.

 

Environment

FileCatalyst Direct Suite v3.4 and later.

 

Resolution

FileCatalyst Direct Server:

  1. Windows

    1. Stand-alone:  When running FileCatalyst Server stand-alone (directly launching the executable), the FileCatalyst Server will run using a maximum of 1024 MB of memory.  This cannot be modified since the values are embedded inside the FCServer.exe binary.

    2. Service:  When running FileCatalyst Server as a Windows Service, you can configure the application to use either a smaller or larger memory footprint based on your needs:
      1. Stop FileCatalyst Server Service.
      2. Open the fcconf.conf found in the application installation directory (by default C:\Program Files\FileCatalyst Server\fcconf.conf) in a text editor.
      3. Locate the following entries in the file:

        # Initial Java Heap Size (in MB)
        wrapper.java.initmemory=1024
        # Maximum Java Heap Size (in MB)
        wrapper.java.maxmemory=4096

      4. Enter the desired variable Initial and Maximum Memory.

    3. Save the file.
    4. Start the FileCatalyst Server Service.

  2. Linux

    1. Stand-alone:  Linux installations include basic startup scripts (fc_server.sh, fc_server_console.sh) which can be modified to use more memory.  
      1. Here is a sample from a launch script with 1GB minimum and 4GB maximum memory (non-continuous block):

        java -Xms1024 -Xmx4096M -jar FileCatalystServer.jar -systray


      2. To run with 4GB of RAM, modify the script to look like the following:

        java -Xms4096M -Xmx4096M -jar FileCatalystServer.jar -systray


    2. Service:  When running FileCatalyst Server as a Linux/Solaris Service, you may configure the application to use a different memory footprint by editing wrapper.conf:
      1. Stop FileCatalyst Server Service.
      2. Edit wrapper.conf found in the application installation directory (by default /opt/utechsoft/server/conf/wrapper.conf) using vi or another text editor.
      3. Enter the memory values you require for the following fields:

        # Initial Java Heap Size (in MB)
        wrapper.java.initmemory=1024
        # Maximum Java Heap Size (in MB)
        wrapper.java.maxmemory=4096

        Choose your values for minimum and maximum memory and enter it.

      4. Save and close the wrapper.conf file.
      5. Start up FileCatalyst Server Service.

  3. MacOSX

    1. MacOSX configurations are managed by an Info.plist file residing inside the FileCatalystServer.app package which tells the system the parameters needed to start the application.
      1. To access the Info.plist, select the FileCatalystServer.app icon and right-click.
      2. Select Show Package Contents.
      3. Edit the info.plist with a text editor.
      4. To increase the memory used edit  the following line:

        <string>-Xms1024M -Xmx4096M</string>

      5. Save the file and close the editor.

FileCatalyst HotFolder

  1. Windows

    1. Stand-alone:  When running FileCatalyst HotFolder stand-alone (directly launching the executable), the FileCatalyst HotFolder will run using a maximum of 1024 MB of memory.  This cannot be modified since the values are embedded inside the FCHotFolder.exe binary.

    2. Service:  When running FileCatalyst HotFolder as a Windows Service, you can configure the application to use either a smaller or larger memory footprint based on your needs:
      1. Stop FileCatalyst HotFolder Service.
      2. Open the fchf.conf found in the application installation directory (by default C:\Program Files\FileCatalyst HotFolder\fchf.conf) in a text editor.
      3. Locate the following entries in the file:

        # Initial Java Heap Size (in MB)
        wrapper.java.initmemory=1024
        # Maximum Java Heap Size (in MB)
        wrapper.java.maxmemory=4096

      4. Enter the desired variable Initial and Maximum Memory.

    3. Save the file.
    4. Start FileCatalyst HotFolder Service.

  2. Linux

    1. Stand-alone:  Linux installations include basic startup scripts (fc_hotfolder.sh, fc_hotfolder_console.sh) which can be modified to use more memory.  Here is a sample from a launch script with 1GB minimum and 4GB maximum memory (non-continuous block):

      java -Xms1024 -Xmx4096M -jar FileCatalystHotFolder.jar -systray


    2. To run with 4GB of RAM, modify the script to look like the following:

      java -Xms4096M -Xmx4096M -jar FileCatalystHotFolder.jar -systray

    3. Service:  When running FileCatalyst HotFolder as a Linux/Solaris Service, you may configure the application to use a different memory footprint by editing wrapper.conf (/opt/utechsoft/hotfolder/conf/wrapper.conf).
    4. Enter the memory values you require for the following fields:

      # Java Additional Parameters
      # Initial Java Heap Size (in MB)
      wrapper.java.initmemory=1024
      # Maximum Java Heap Size (in MB)
      wrapper.java.maxmemory=4096

      Choose your values for minimum and maximum memory and enter it.
    5. Save and close the wrapper.conf file.
    6. Start up FileCatalyst HotFolder Service.

  3. Mac OSX

    1. MacOSX configurations are managed by an Info.plist file residing inside the FileCatalystHotFolder.app package which tells the system the parameters needed to start the application.
      1. To access the Info.plist, select the FileCatalystHotFolder.app icon and right-click.
      2. Select Show Package Contents.
      3. Edit the Info.plist with a text editor.
      4. To increase the memory used edit  the following line:

        <string>-Xms64M</string>
        <string>-Xmx4096M</string>
      5. Save the file and close the editor.

FileCatalyst Central

  1. Windows

    1. From the Service Manager application (Services.msc), stop the FileCatalyst Central Service.
    2. Locate the file mawrapper.conf in the FileCatalyst Central installation and open it in a text editor.

    3. To increase the memory used, edit the following lines:

      # Initial Java Heap Size (in MB)
      wrapper.java.initmemory=1024
      # Maximum Java Heap Size (in MB)
      wrapper.java.maxmemory=4096

      To maximize performance improvement we recommend that both values be identical, which will cause Java to allocate a contiguous block of memory.

    4. Close the text editor and start the FileCatalyst Central Service.

  2. Linux
    1. Stop the FileCatalyst Central Service using:

      service fc_central stop

    2. Locate the file wrapper.conf in the FileCatalyst Central installation directory and within the /opt/utechsoft/central/conf/wrapper.conf directory and open it in a text editor.
    3. To increase memory used by Central, edit the following lines:

      # Initial Java Heap Size (in MB)
      wrapper.java.initmemory=1024
      # Maximum Java Heap Size (in MB)
      wrapper.java.maxmemory=4096

      To maximize performance improvement we recommend that both values be identical, which will cause Java to allocate a contiguous block of memory.

    4. Save the file and close the text editor.

    5. Start the FileCatalyst Central Service using the command:

      service fc_central start