Rosetta Load Balancer Example
Load Balancer Configuration:
Load balancing allows distribution of workloads across multiple Rosetta servers. Such configuration aims to optimize resource use, maximize throughput, minimize response time, and avoid overload of a single server.
In addition, configuring load balancing can provide SSL solutions for Rosetta. If you load balance on the HTTPS layer, then you’d install the SSL certificate on the load balancer alone, and use plain un-encrypted HTTP over the local network between the load balancer and the Rosetta servers.
Load Balancers can be configured on the hardware level (e.g F5) or on the software level (e.g Apache mod_ proxy). We will be using the Apache software solution for this example.
Rosetta Load Balancer Example:
For the purpose of this example I am using the following:
- Two Rosetta servers – rosetta01.myinstitution.org and rosetta02.myinstitution.org. Both are “all in one”servers (DEP, REP, PER, DEL and IDX roles)
- A Load Balancer machine: loadbalancer.myinstitution.org. Apache has been installed with mod_proxy and mod_ssl on this server under /httpd/
Before starting this procedure, shut down Apache (LB and PDS) and Rosetta.
The configuration consists of two parts: configuring the Rosetta servers and configuring the Load Balancer machine:
Rosetta Side Configuration:
You will need to edit the relevant properties in the global.properties file on all servers. In this example I am using SSL (port 443) for PDS requests only. I have updated the following global.properties on both rosetta01.myinstitution.org and rosetta02.myinstitution.org :
load.balancer.dep.port=80 load.balancer.del.port=80 load.balancer.rep.port=80 load.balancer.per.port=80 load.balancer.pds.port=443 load.balancer.dep.host=loadbalancer.myinstitution.org load.balancer.del.host=loadbalancer.myinstitution.org load.balancer.rep.host=loadbalancer.myinstitution.org load.balancer.per.host=loadbalancer.myinstitution.org load.balancer.pds.host=loadbalancer.myinstitution.org load.balancer.dep=loadbalancer.myinstitution.org\:80 load.balancer.del=loadbalancer.myinstitution.org\:80 load.balancer.rep=loadbalancer.myinstitution.org\:80 load.balancer.per=loadbalancer.myinstitution.org\:80 load.balancer.pds=loadbalancer.myinstitution.org\:443
We have set all hosts to our LB machine with the standard https port for pds and the standard http port for other hosts.
If you are using SSL for PDS, I will explain the changes necessary at this point in the PDS module later in this blog.
Now all that’s left to do is Run dps_config and we are done with the Rosetta side configuration.
Load Balancer Machine Configuration:
In order to configure the LB you’ll need to edit the /httpd/conf/httpd.conf file for regular LB configuration Or the /httpd/conf/extra/httpd-ssl.conf file for SSL LB configuration
For SSL Load Balancing you will need to include the httpd-ssl.conf in the httpd.conf file by adding the following line:
You need to define the ports the LB will listen to in our case:
Listen 80 http
Listen 443 https
Define the proxy definitions and rules. For example, our PDS is located on rosetta01.myinstitution.org . Therefore, we will reroute all PDS http request received by the LB to there.
Proxy definition for PDS:
<Proxy balancer://pds> Order deny,allow Allow from all BalancerMember https://rosetta01.myinstitution.org.corp.exlibrisgroup.com:443/ </Proxy>
ProxyPassReverse / https://rosetta01.myinstitution.org.corp.exlibrisgroup.com:443/
Proxy rule for PDS:
RewriteEngine On RewriteRule ^/pds(.*) balancer://pds/pds$1 [P] RewriteRule ^/goto(.*) balancer://pds/goto$1 [P] RewriteRule ^/calling_system-dps(.*) balancer://pds/calling_system-dps$1 [P] ProxyPassReverse / https://rosetta01.myinstitution.org.corp.exlibrisgroup.com:443/
You must define a sticky session for proxy rules that reroute to more than one server. This will insure that all requests will be redirected to the same Rosetta server during the same session. The following configuration redirects all requests to one of our Rosetta servers using a sticky session:
Proxy definition for Rosetta requests:
# staging end user
<Proxy balancer://staging-user> Order deny,allow Allow from all BalancerMember http://rosetta01.myinstitution.org:1801 route=rosetta01.myinstitution.org:1801 BalancerMember http://rosetta02.myinstitution.org:1801 route=rosetta02.myinstitution.org.corp.exlibrisgroup.com:1801 ProxySet stickysession=JSESSIONID lbmethod=byrequests nofailover=On </Proxy>
Proxy rule for Rosetta Requests (everything not caught by the pds proxy rule):
RewriteEngine On RewriteRule ^(.*)$ balancer://staging-user$1 [P] # ProxyPass / balancer://staging-user/ stickysession=JSESSIONID lbmethod=byrequests nofailover=On ProxyPassReverse / https://rosetta01.myinstitution.org:1801 ProxyPassReverse / https://rosetta02.myinstitution.org:1801
Now all you need to do is start the Load Balancer and you are set to go!
Starting and stopping the Apache Load balancer is done by apachectl start or the apachectl stop commands located under the /httpd/bin directory.
SSL LB Configuration:
For SSL you will need a private key (server.key) and a signed certificate (server.crt).
For our example the following steps were necessary:
- Copy the key and the certificate to apache conf: /httpd/conf
- In order to redirect http requests to https, create a virtual Host that listens to port 80 and redirects to https. This is done in the /httpd/conf/extra/httpd.conf file:
<VirtualHost _default_:80> ServerName loadbalancer.myinstitution.org Redirect / https://loadbalancer.myinstitution.org/mng </VirtualHost>
Now create a virtual host that listens to port 443 for all LB redirections to Rosetta. Here is where we define the proxy definitions and rules described in the Load Balancer configuration section of this document.
<VirtualHost _default_:443> ServerName rosetta01.myinstitution.org SSLEngine on SSLProxyEngine On SSLCertificateFile /httpd/conf/server.crt SSLCertificateKeyFile /httpd/conf/server.key <IfModule proxy_module> <Proxy balancer://pds> Order allow,deny Allow from all BalancerMember https://rosetta01.myinstitution.org </Proxy> RewriteEngine On RewriteRule ^/pds(.*) balancer://pds/pds$1 [P] RewriteRule ^/goto(.*) balancer://pds/goto$1 [P] RewriteRule ^/calling_system-dps(.*) balancer://pds/calling_system-dps$1 [P] ProxyPassReverse / https://rosetta01.myinstitution.org/ # staging end user <Proxy balancer://staging-user> Order allow,deny Allow from all BalancerMember http://rosetta01.myinstitution.org:1801 route=rosetta01.myinstitution.org:1801 BalancerMember http://rosetta02.myinstitution.org:1801 route=rosetta02.myinstitution.org:1801 ProxySet stickysession=JSESSIONID lbmethod=byrequests nofailover=On </Proxy> RewriteRule ^(.*)$ balancer://staging-user$1 [P] # ProxyPass / balancer://staging-user/ stickysession=JSESSIONID lbmethod=byrequests nofailover=On ProxyPassReverse / http://rosetta01.myinstitution.org:1801 ProxyPassReverse / http://rosetta02.myinstitution.org:1801 </IfModule> </VirtualHost>
SSL PDS Configuration:
- Go to pdsroot/program/PDSDefinitions and change the port to 443 and the protocol to https:
our ($server_httpd) = "loadbalancer.myinstitution.org:443"; our ($server_httpsd) = "loadbalancer.myinstitution.org:443"; our ($server_pds) = "loadbalancer.myinstitution.org:443/pds";
- You will need to edit the relevant properties in the global.properties file on all servers. on both rosetta01.myinstitution.org and rosetta02.myinstitution.org :
load.balancer.dep.port=443 load.balancer.del.port=443 load.balancer.rep.port=443 load.balancer.per.port=443 load.balancer.pds.port=443 load.balancer.dep=loadbalancer.myinstitution.org\:443 load.balancer.del=loadbalancer.myinstitution.org\:443 load.balancer.rep=loadbalancer.myinstitution.org\:443 load.balancer.per=loadbalancer.myinstitution.org\:443 protocol.del=https protocol.dep=https protocol.pds=https protocol.per=https protocol.rep=https
- Place your SSL key and certificate under pdsroot/apache/SSLconf/conf
- Make sure -DSSL is set in the pdsroot/apache/bin/apachectl_auto:
$httpd_bin/httpd -d $httpd_root -DSSL
- Make sure “User” is set to dps and”Group” is set to exlibris under pdsroot/apache/conf/httpd.conf
- “su” to root, and run ./apachectl_stop and ./apachectl_auto under pdsroot/apache/bin/
- Update tab_services (e.g pdsroot/conf_table/tab_service.INS00). Set port to 443 and SSL to “Y” for yes.
The code is also available on github.